decompress—a wrapper script to decompress various archive types

00-post

The Arch Linux BBS has a thread where people put up their scripts so that others can peruse them. A long time ago someone came up with the idea to create a script that would detect various archive formats and decompress them. That post is unfortunately gone now, but I kept the idea and have expanded on it a bit: I’ve added a couple archive types, file detection, program detection, and archive list support. I gave it a good, overall test so I feel comfortable with it.

Options can be in any order:

$ decompress archive-r.zip --help
decompress [*-l] ... — wrapper script to decompress various archive types
  -l, --list  - list archive contents

If an archive’s existence isn’t detected it will be displayed:

$ decompress archive-r.zip
archive non-existent: archive-r.zip

If a program’s existence isn’t detected it will be displayed:

$ decompress archive-q.zip
program required: unzip

Listing support is available:

$ decompress -l archive-q.zip
 archive-q.zip
       32  2016-04-11 10:39   file-q1
       32  2016-04-11 10:39   file-q2

Listing and decompressing can be done for multiple documents:

$ ls
archive-a.tar.bz2  archive-f.tgz       archive-k.txz  archive-p.xz
archive-b.tb2      archive-g.tar.lz    archive-l.7z   archive-q.zip
archive-c.tbz      archive-h.tar.lzma  archive-m.bz2
archive-d.tbz2     archive-i.tlz       archive-n.gz
archive-e.tar.gz   archive-j.tar.xz    archive-o.lz
$ decompress archive-*
archive-a.tar.bz2...
archive-b.tb2...
archive-c.tbz...
archive-d.tbz2...
archive-e.tar.gz...
archive-f.tgz...
archive-g.tar.lz...
archive-h.tar.lzma...
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
archive-i.tlz...
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
archive-j.tar.xz...
archive-k.txz...
archive-l.7z...
archive-m.bz2...
archive-n.gz...
archive-o.lz...
archive-p.xz...
archive-q.zip...

.exe and .rar files are untested because I was lazy. If there is an error its error message will be displayed.

decompress can be found in my general-scripts repository.

Advertisements

compress—a tar wrapper script to simplify archiving files

00-post

I have become accustomed to using long options over the years as they are easier to remember. I do however use tar in numerous ways. I needed to have a quick way to remember how to archive files; I wrote this script to make it real basic:

$ cd ~
$ compress .local/bin/ Development/general-scripts/
archive name [archive.tar.gz]: /dev/sda4/sc
scripture.css  scripts.tar.xz
archive name [archive.tar.gz]: /dev/sda4/scripts.tar.xz
archive exists, overwrite? (y/n): y
archive created: scripts.tar.xz

The compression type to be used will depend on which extension is typed; tar has a nice option called --auto-compress. So, in the above example, typing ...tar.xz will use the LZMA compression algorithm. Just typing Enter on the archive name and the default archive.tar.gz will be used. The script also supports tab-completion for typing the archive name to help navigate folders and files.

compress can be found in my general-scripts repository.

Perl module installation

If doing Perl programming or if another package requires a Perl module, learning how to install one may become necessary. The recommended way to install a Perl module is through the distribution’s repositories, however, they can be installed manually with Perl.

Configure

Perl has its own repository where programmers make available their modules called the comprehensive Perl archive network, which is better known as the CPAN. Perl includes a built-in module that can download, build, and install from the network. For some distributions this module may already be built, however, it is probably a good idea for all to build it… to be sure it is set up correctly. Begin by starting the CPAN module shell so that it may be configured:

perl -MCPAN -e shell

A configuration message will appear… most users will be good with the automatic configuration it recommends. If additional configuring needs to be done later typing o conf init will re-run the configuration dialog. To leave the shell type exit.

Install

The first requirement most people will need to do is build and/or update the CPAN module. Modules can be installed with the built-in module in three ways: from the module shell, from the perl command, or from the CPAN module binary.

From the shell (which was entered in the configuration section), the following command will install a new module, or in this case, update the CPAN module:

install Bundle::CPAN

From the perl command:

perl -MCPAN -e 'install HTML::Template'

From the cpan module binary:

cpan Module::Name

Note: CPAN itself recommends using the cpanm module for installation. Modules will need to be reloaded after being updated: reload cpan.

Execute

Modules are sometimes executable binaries and if they are known to the shell can be executed like any other command. Some modules are support modules and can only be used for programming or by use of another module. Information of installed modules can be discovered with the command perldoc perllocal.

Uninstall

Module maintenance is typically unexpected after installation and the built-in CPAN module has no ability to be able to do so. If the cpanm module is installed it does have the ability with the --uninstall/-U option. It will display the files to be removed and prompt for approval before uninstalling.

System backup to DVD

The purpose of this article is to explain how to span large archives into multiple files. One would do this, for example, to store on numerous DVDs.

Rational

I had an occurrence where the only form of storage media I had were DVDs. It became necessary to create a full operating system backup and I was able to do so with the DVDs. (A system backup with DVDs is atypical because DVDs have dyes which have limited time usage.) The following explanation is for an Operating System multi-volume archive creation that can also be used on other large archives.

Operating System archive creation

From an Install CD I did my archive creation. Such an archive will nearly always need to be done from such an external operating system (a running operating system is always adding or editing files that necessitates using an external operating system).

My disk partitions I label according to the Operating System. The archive name I used begins with my hostname which is my computer model:

mkdir               /mnt/LABEL
mount /dev/disk/by-label/LABEL
cd                  /mnt/LABEL
archive_name=$HOSTNAME_$(date +%F)_DISTRO.tar.xz
tar cavf $archive_name .

Archive Multi-voluming

I can now divide the archive to multiple volumes. I do this so that they fit on a DVD. A few explanations first:

  • DVD storage capacity varies. For me I used a 4.7 Gigabyte DVD.
  • DVD storage capacity will likely need to be converted so a multi-volume file will fit on a DVD. DVD storage capacity is almost always calculated with metric prefixes (i.e. a base of 1000); however, typical computer numerology uses the binary prefixes a base of 1024. The metric 4.7 Gigabyte converted to binary is 4.37721 Gibibyte (GB to GiB conversion fraction: 1,000,000,000/[1024 x 1024 x 1024] = 0.9313226).
  • The UDF file system, typically used for DVD data storage, at the time of this writing was still experimental on Linux and I choose to use the ISO-9660 file system for reliability. However, this file system has a file size limit of four GiB. Since my writable DVDs had a capacity of 4.7 GB I had to split up the files to two per DVD. File system overhead also has to be factored in and I reduced 2.188608100 GiB to 2.188000 GiB.

There are two methods that typically do this:

split

I default to the split command because the open source philosophy “do one job and do it good”. It is easy to operate: -b/--bytes= is converted to Mebibytes and -d appends a numerical suffix. (Notice the period following to the second value.)

split -b 2240M -d HOSTNAME_DATE_DISTRO.tar.xz HOSTNAME_DATE_DISTRO.tar.xz.

tar

tar can also do it. Gibibytes is converted to Kibibytes (1024 bits). To _c_reate a _L_ength-limited _M_ulti-volume archive:

tar -cML 2284284 -f HOSTNAME_DATE_DISTRO.tar.xz.00 HOSTNAME_DATE_DISTRO.tar.xz

The first multi-volume file will be created and then the name of the next file will be requested:

Prepare volume #2 for HOSTNAME_DATE_DISTRO.tar.xz.00
# n HOSTNAME_DATE_DISTRO.tar.xz.01

If the save location is in another location the path will need to be entered on every new entry (e.g. n /mnt/Backup/HOSTNAME_DATE_DISTRO.tar.xz.02).

DVD writing

I believe growisofs is a good way to do this:

growisofs -Z /dev/dvd -rJ HOSTNAME_DATE_DISTRO.tar.xz.{00,01}

Operating System archive restore

Boot from the Install CD, then:

mkdir /mnt/LABEL
mount /dev/dvd /mnt/dvd
mount /dev/disk/by-label/LABEL /mnt/LABEL

File system creation I am assuming is already done. To join the multi-volume archives (if done by split and tar respectively):

cat      HOSTNAME_DATE_DISTRO.tar.xz.*  > HOSTNAME_DATE_DISTRO.tar.xz
tar -xMf HOSTNAME_DATE_DISTRO.tar.xz.00   HOSTNAME_DATE_DISTRO.tar.xz

Then the archive will just need decompressing:

tar xvf HOSTNAME_DATE_DISTRO.tar.xz -C /mnt/LABEL

External links

Patterns and textures

These patterns were originally for Nautilus, the GNOME file manager (now called “Files”), when it supported having images as backgrounds. They are still good for some other programs though so I touched them up a bit.

I designed these so readability was good with them which was the key factor. A background I feel is to help with the comfort-ability of the area. They are barely noticed and add grip to the area. An original example:

screenshot-jpasd-file-browser.png

And here is a view of them all:

example

Enjoy!

Firefox profile on a flash drive

firefox-tuning

I regularly use a computer at my workplace. This computer multiple people use so it is setup not to save Firefox’s settings. I use it enough, in specific ways, that I decided to find a way to use it with settings available.

I take a flash drive with me to be able to access various personal documents and programs. I had heard about web browsers being installed on flash drives; after I thought about it awhile, I realized all I would be required to do is put a profile on it.

The flash drive I have is formatted FAT32 to be able to use with Windows (my work computer) and I keep my flash drive organized similar to my Linux home directory for convenience. First I created a directory for the profile:

mkdir -p ~/.mozilla/firefox/profile/ANAME

Then I create the profile—from the command prompt this command will get the job done:

start firefox.exe -CreateProfile "MyName D:\.mozilla\firefox\profile\ANAME"

start-menu-example

(Or alternately I could have typed firefox.exe -P from the start menu and used the GUI version.)

I started the profile then to have the necessary files created. After it got done loading, I quit Firefox and I deleted the profile managers knowledge of the profile but not the profile itself (see example picture).

With the profile created all required effort left to be done is to instruct Firefox of the profile’s location. I put this in a batch script so that I can regularly use it:

@echo off

:: http://stackoverflow.com/a/15815897
:: http://stackoverflow.com/q/154075

set HOMEDRIVE=%cd:~0,2%

if exist C:\PATH\TO\firefox.exe start /b C:\PATH\TO\firefox.exe -profile %HOMEDRIVE%\.mozilla\firefox\profile\toddweed && exit

if exist C:\PATH\TO\firefox.exe start /b C:\PATH\TO\firefox.exe -profile %HOMEDRIVE%\.mozilla\firefox\profile\toddweed && exit

if exist C:\Program Files\Mozilla Firefox\firefox.exe (
  start /b C:\PATH\TO\firefox.exe -profile %HOMEDRIVE%\.mozilla\firefox\profile\toddweed
) else (
  echo Firefox executable not found.
)

GNU love for Windows

gwindows_logo

I wanted to be able to type in Windows with a familiar text editor. I downloaded a terminal text editor called gvim, which I think is a good text editor, and it was able to be installed portably—this last is necessary as the computer I use I am not able to install anything on. However, even with having that, I discovered that I wanted to be to use the GNU tools I had become familiar with… hence a dilemma.

There are several projects that provide GNU utilities on Windows… I have learned. I am not an expert in these mind you—I have tried only one—however, I’ve heard good things about them all. The following are all terminal emulators and they include the GNU utilities: Cygwin, Gow (GNU on Windows), and Git-bash.

I have only tried the later one. The reason for this is because I’ve had it already installed as it comes installed with the Git program that I use at times. I would have like to tried for the first two, however I’m not too picky and Git-bash has done me well enough; it has the basic utilities and is pretty much ready to go.

Having GNU utilities available is handy for me and saves me a bit of time because of familiarity; another bonus is time saved that would be required to learn the Windows command line. The tools run just like they do on Linux/Unix and can be used on the whole file system. Many of the tools are there: sed, awk, mkdir…. For example, I can type:

$ find /c/Windows/Web/ -type f -name "*Think*"
/c/Windows/Web/Wallpaper/Think/Think_Black.jpg
/c/Windows/Web/Wallpaper/Think/Think_Blue.jpg

By default the terminal emulator uses the already set Windows %HOMEPATH% variable for shell’s $HOME directory—this is usually C:\Users\USERNAME. I decided to use my flash drive as the $HOME directory where all my documents and settings could be kept. I had to create a batch script that defined the %HOMEPATH% and then have it start Git-bash:

@echo off

set HOMEDRIVE=%cd:~0,2%
set HOMEPATH=\

start /B %HOMEDRIVE%\Downloads\Git\git-bash.exe

01-git-bash-example

I then created a shortcut to flash drive root directory for quick access and to have a custom icon. I icon I choose was taken from the git-bash.exe file when I was asked for the icon location.

Git setup

An error was the first thing I had to fix… and it may just be for my particular version of Git; it complained to me when I tried to use it and I had to specify the certificate location. I did this by:

git config --system http.sslcainfo /d/Downloads/Git/mingw32/ssl/certs/ca-bundle.crt

After this I added ssh-agent to my ~/.bashrc launch it:

# SSH agent auto-launch
# 0 = agent running with key; 1 = agent running w/o key; 2 = agent not running
agent_run_state=$(ssh-add -l >| /dev/null 2>&1; echo $?)
if   [ $agent_run_state = 2 ]; then
  eval $(ssh-agent -s)
  ssh-add
elif [ $agent_run_state = 1 ]; then
  ssh-add
fi

And likewise I added to the bash_logout file ssh-agent -k as Windows would think that it was still running if I didn’t.

Now I’m working pretty good in Windows.