decompress—a wrapper script to decompress various archive types

00-post

The Arch Linux BBS has a thread where people put up their scripts so that others can peruse them. A long time ago someone came up with the idea to create a script that would detect various archive formats and decompress them. That post is unfortunately gone now, but I kept the idea and have expanded on it a bit: I’ve added a couple archive types, file detection, program detection, and archive list support. I gave it a good, overall test so I feel comfortable with it.

Options can be in any order:

$ decompress archive-r.zip --help
decompress [*-l] ... — wrapper script to decompress various archive types
  -l, --list  - list archive contents

If an archive’s existence isn’t detected it will be displayed:

$ decompress archive-r.zip
archive non-existent: archive-r.zip

If a program’s existence isn’t detected it will be displayed:

$ decompress archive-q.zip
program required: unzip

Listing support is available:

$ decompress -l archive-q.zip
 archive-q.zip
       32  2016-04-11 10:39   file-q1
       32  2016-04-11 10:39   file-q2

Listing and decompressing can be done for multiple documents:

$ ls
archive-a.tar.bz2  archive-f.tgz       archive-k.txz  archive-p.xz
archive-b.tb2      archive-g.tar.lz    archive-l.7z   archive-q.zip
archive-c.tbz      archive-h.tar.lzma  archive-m.bz2
archive-d.tbz2     archive-i.tlz       archive-n.gz
archive-e.tar.gz   archive-j.tar.xz    archive-o.lz
$ decompress archive-*
archive-a.tar.bz2...
archive-b.tb2...
archive-c.tbz...
archive-d.tbz2...
archive-e.tar.gz...
archive-f.tgz...
archive-g.tar.lz...
archive-h.tar.lzma...
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
archive-i.tlz...
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
archive-j.tar.xz...
archive-k.txz...
archive-l.7z...
archive-m.bz2...
archive-n.gz...
archive-o.lz...
archive-p.xz...
archive-q.zip...

.exe and .rar files are untested because I was lazy. If there is an error its error message will be displayed.

decompress can be found in my general-scripts repository.

compress—a tar wrapper script to simplify archiving files

00-post

I have become accustomed to using long options over the years as they are easier to remember. I do however use tar in numerous ways. I needed to have a quick way to remember how to archive files; I wrote this script to make it real basic:

$ cd ~
$ compress .local/bin/ Development/general-scripts/
archive name [archive.tar.gz]: /dev/sda4/sc
scripture.css  scripts.tar.xz
archive name [archive.tar.gz]: /dev/sda4/scripts.tar.xz
archive exists, overwrite? (y/n): y
archive created: scripts.tar.xz

The compression type to be used will depend on which extension is typed; tar has a nice option called --auto-compress. So, in the above example, typing ...tar.xz will use the LZMA compression algorithm. Just typing Enter on the archive name and the default archive.tar.gz will be used. The script also supports tab-completion for typing the archive name to help navigate folders and files.

compress can be found in my general-scripts repository.

Perl module installation

If doing Perl programming or if another package requires a Perl module, learning how to install one may become necessary. The recommended way to install a Perl module is through the distribution’s repositories, however, they can be installed manually with Perl.

Configure

Perl has its own repository where programmers make available their modules called the comprehensive Perl archive network, which is better known as the CPAN. Perl includes a built-in module that can download, build, and install from the network. For some distributions this module may already be built, however, it is probably a good idea for all to build it… to be sure it is set up correctly. Begin by starting the CPAN module shell so that it may be configured:

perl -MCPAN -e shell

A configuration message will appear… most users will be good with the automatic configuration it recommends. If additional configuring needs to be done later typing o conf init will re-run the configuration dialog. To leave the shell type exit.

Install

The first requirement most people will need to do is build and/or update the CPAN module. Modules can be installed with the built-in module in three ways: from the module shell, from the perl command, or from the CPAN module binary.

From the shell (which was entered in the configuration section), the following command will install a new module, or in this case, update the CPAN module:

install Bundle::CPAN

From the perl command:

perl -MCPAN -e 'install HTML::Template'

From the cpan module binary:

cpan Module::Name

Note: CPAN itself recommends using the cpanm module for installation. Modules will need to be reloaded after being updated: reload cpan.

Execute

Modules are sometimes executable binaries and if they are known to the shell can be executed like any other command. Some modules are support modules and can only be used for programming or by use of another module. Information of installed modules can be discovered with the command perldoc perllocal.

Uninstall

Module maintenance is typically unexpected after installation and the built-in CPAN module has no ability to be able to do so. If the cpanm module is installed it does have the ability with the --uninstall/-U option. It will display the files to be removed and prompt for approval before uninstalling.

System backup to DVD

Splitting Files

The purpose of this article is to detail how to backup a Linux system using a DVD writer; this will likely take multiple DVDs.

Rational

A backup can be done to multiple DVDs in situations where it is necessary. However, keep in mind DVDs can be damaged and that the organic dye in DVDs deteriorates over time… backing up to external hard drives is recommended.

System backup

Start an install CD or other likewise media to be able to access the drive/partition without system operation interference. Backups must be done from an external medium as files are constantly written to on a live system.

mkdir /mnt/distro
mount /dev/sd[#] /mnt/distro
cd    /mnt/distro
tar   -czvf distro_`date +%F`.tar.gz .  # (c)reate (z)ip (v)erbose (f)ile

Backup split

The backup can be split into multiple files so they will fit on a DVD.

Note 1: A DVD’s storage capacity may be smaller then some expect. The reason is because the original computer developers wanted to use binary as the base storage measurement (a kilobyte to be 1024 bits) and erred when they adopted metric prefixes. Data storage manufacturers began using these 1000 base prefixes because they displayed larger values. A 4.7 GB DVD in binary units with the later and gradually being adopted binary prefix is 4.37721 GiB— this is the number that computer veterans typically expect. (GB to GiB conversion fraction: 1,000,000,000/[1024 * 1024 * 1024] = 0.9313226)

Note 2: The UDF file system, typically used for DVD data storage, at the time of this writing was still experimental on Linux and I choose to use the ISO-9660 file system for reliability. However, this file system has a file size limit of four GiB. Since my writable DVDs had a capacity of 4.7 GB I had to split up the files to two per DVD. File system overhead also has to be factored in and I reduced 2.188608100 GiB to 2.188000 GiB.

tar usage

With tar create a Length-defined, Multi-volume archive (length is in 1 KiB units):

tar -cML 2188000 -f distro_`date +%F`.tar.gz_00 distro_`date +%F`.tar.gz

After the first segment is split a prompt will request the name of the next file (Prepare volume #2 for 'distro_DATE.tar.gz_00):

n distro_DATE_tar.gz_01

(This command, tar and multi-voluming, can also be done from the beginning of the backup if archive compression is unnecessary.)

split usage

Splitting can also be done with the split command. Here 2.188000 GiB is converted to 2240.512 MiB and -d adds a numerical suffix:

split -b 2240M -d distro_`date +%F`.tar.gz distro_`date +%F`.tar.gz_

DVD write

To use growisofs to burn the files to a writable DVD disc:

growisofs -Z /dev/dvd -rJ distro_`date +%F`_00.tar.gz distro..._01.tar.gz

System restore

Boot from the Install CD or other boot medium (boot with a caching option if available in the boot menu if another DVD reader is unavailable).

$ umount /dev/dvd && eject
mount /dev/dvd /mnt/dvd
mkdir /mnt/distro

Create the file system on the drive/partition if necessary and mount it.

Backup join

With tar:

tar -xMf distro_`date +%F`.tar.gz_00 distro_`date +%F`.tar.gz

With cat (for the split files):

cat distro_`date +%F`.tar.gz_* > distro_`date +%F`.tar.gz

Backup restore

tar xvf distro_`date +%F`.tar.gz -C /mnt/distro

Resources

Patterns and textures

These patterns were originally for Nautilus, the GNOME file manager (now called “Files”), when it supported having images as backgrounds. They are still good for some other programs though so I touched them up a bit.

I designed these so readability was good with them which was the key factor. A background I feel is to help with the comfort-ability of the area. They are barely noticed and add grip to the area. An original example:

screenshot-jpasd-file-browser.png

And here is a view of them all:

example

Enjoy!

Firefox profile on a flash drive

firefox-tuning

I regularly use a computer at my workplace. This computer multiple people use so it is setup not to save Firefox’s settings. I use it enough, in specific ways, that I decided to find a way to use it with settings available.

I take a flash drive with me to be able to access various personal documents and programs. I had heard about web browsers being installed on flash drives; after I thought about it awhile, I realized all I would be required to do is put a profile on it.

The flash drive I have is formatted FAT32 to be able to use with Windows (my work computer) and I keep my flash drive organized similar to my Linux home directory for convenience. First I created a directory for the profile:

mkdir -p ~/.mozilla/firefox/profile/ANAME

Then I create the profile—from the command prompt this command will get the job done:

start firefox.exe -CreateProfile "MyName D:\.mozilla\firefox\profile\ANAME"

start-menu-example

(Or alternately I could have typed firefox.exe -P from the start menu and used the GUI version.)

I started the profile then to have the necessary files created. After it got done loading, I quit Firefox and I deleted the profile managers knowledge of the profile but not the profile itself (see example picture).

With the profile created all required effort left to be done is to instruct Firefox of the profile’s location. I put this in a batch script so that I can regularly use it:

@echo off

:: http://stackoverflow.com/a/15815897
:: http://stackoverflow.com/q/154075

set HOMEDRIVE=%cd:~0,2%

if exist C:\PATH\TO\firefox.exe start /b C:\PATH\TO\firefox.exe -profile %HOMEDRIVE%\.mozilla\firefox\profile\toddweed && exit

if exist C:\PATH\TO\firefox.exe start /b C:\PATH\TO\firefox.exe -profile %HOMEDRIVE%\.mozilla\firefox\profile\toddweed && exit

if exist C:\Program Files\Mozilla Firefox\firefox.exe (
  start /b C:\PATH\TO\firefox.exe -profile %HOMEDRIVE%\.mozilla\firefox\profile\toddweed
) else (
  echo Firefox executable not found.
)

GNU love for Windows

gwindows_logo

I wanted to be able to type in Windows with a familiar text editor. I downloaded a terminal text editor called gvim, which I think is a good text editor, and it was able to be installed portably—this last is necessary as the computer I use I am not able to install anything on. However, even with having that, I discovered that I wanted to be to use the GNU tools I had become familiar with… hence a dilemma.

There are several projects that provide GNU utilities on Windows… I have learned. I am not an expert in these mind you—I have tried only one—however, I’ve heard good things about them all. The following are all terminal emulators and they include the GNU utilities: Cygwin, Gow (GNU on Windows), and Git-bash.

I have only tried the later one. The reason for this is because I’ve had it already installed as it comes installed with the Git program that I use at times. I would have like to tried for the first two, however I’m not too picky and Git-bash has done me well enough; it has the basic utilities and is pretty much ready to go.

Having GNU utilities available is handy for me and saves me a bit of time because of familiarity; another bonus is time saved that would be required to learn the Windows command line. The tools run just like they do on Linux/Unix and can be used on the whole file system. Many of the tools are there: sed, awk, mkdir…. For example, I can type:

$ find /c/Windows/Web/ -type f -name "*Think*"
/c/Windows/Web/Wallpaper/Think/Think_Black.jpg
/c/Windows/Web/Wallpaper/Think/Think_Blue.jpg

By default the terminal emulator uses the already set Windows %HOMEPATH% variable for shell’s $HOME directory—this is usually C:\Users\USERNAME. I decided to use my flash drive as the $HOME directory where all my documents and settings could be kept. I had to create a batch script that defined the %HOMEPATH% and then have it start Git-bash:

@echo off

set HOMEDRIVE=%cd:~0,2%
set HOMEPATH=\

start /B %HOMEDRIVE%\Downloads\Git\git-bash.exe

01-git-bash-example

I then created a shortcut to flash drive root directory for quick access and to have a custom icon. I icon I choose was taken from the git-bash.exe file when I was asked for the icon location.

Git setup

An error was the first thing I had to fix… and it may just be for my particular version of Git; it complained to me when I tried to use it and I had to specify the certificate location. I did this by:

git config --system http.sslcainfo /d/Downloads/Git/mingw32/ssl/certs/ca-bundle.crt

After this I added ssh-agent to my ~/.bashrc launch it:

# SSH agent auto-launch
# 0 = agent running with key; 1 = agent running w/o key; 2 = agent not running
agent_run_state=$(ssh-add -l >| /dev/null 2>&1; echo $?)
if   [ $agent_run_state = 2 ]; then
  eval $(ssh-agent -s)
  ssh-add
elif [ $agent_run_state = 1 ]; then
  ssh-add
fi

And likewise I added to the bash_logout file ssh-agent -k as Windows would think that it was still running if I didn’t.

Now I’m working pretty good in Windows.