Author Archives: Todd Partridge

About Todd Partridge

Society first.

decompress—a wrapper script to decompress various archive types

00-post

The Arch Linux BBS has a thread where people put up their scripts so that others can peruse them. A long time ago someone came up with the idea to create a script that would detect various archive formats and decompress them. That post is unfortunately gone now, but I kept the idea and have expanded on it a bit: I’ve added a couple archive types, file detection, program detection, and archive list support. I gave it a good, overall test so I feel comfortable with it.

Options can be in any order:

$ decompress archive-r.zip --help
decompress [*-l] ... — wrapper script to decompress various archive types
  -l, --list  - list archive contents

If an archive’s existence isn’t detected it will be displayed:

$ decompress archive-r.zip
archive non-existent: archive-r.zip

If a program’s existence isn’t detected it will be displayed:

$ decompress archive-q.zip
program required: unzip

Listing support is available:

$ decompress -l archive-q.zip
 archive-q.zip
       32  2016-04-11 10:39   file-q1
       32  2016-04-11 10:39   file-q2

Listing and decompressing can be done for multiple documents:

$ ls
archive-a.tar.bz2  archive-f.tgz       archive-k.txz  archive-p.xz
archive-b.tb2      archive-g.tar.lz    archive-l.7z   archive-q.zip
archive-c.tbz      archive-h.tar.lzma  archive-m.bz2
archive-d.tbz2     archive-i.tlz       archive-n.gz
archive-e.tar.gz   archive-j.tar.xz    archive-o.lz
$ decompress archive-*
archive-a.tar.bz2...
archive-b.tb2...
archive-c.tbz...
archive-d.tbz2...
archive-e.tar.gz...
archive-f.tgz...
archive-g.tar.lz...
archive-h.tar.lzma...
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
archive-i.tlz...
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
archive-j.tar.xz...
archive-k.txz...
archive-l.7z...
archive-m.bz2...
archive-n.gz...
archive-o.lz...
archive-p.xz...
archive-q.zip...

.exe and .rar files are untested because I was lazy. If there is an error its error message will be displayed.

decompress can be found in my general-scripts repository.

Advertisements

compress—a tar wrapper script to simplify archiving files

00-post

I have become accustomed to using long options over the years as they are easier to remember. I do however use tar in numerous ways. I needed to have a quick way to remember how to archive files; I wrote this script to make it real basic:

$ cd ~
$ compress .local/bin/ Development/general-scripts/
archive name [archive.tar.gz]: /dev/sda4/sc
scripture.css  scripts.tar.xz
archive name [archive.tar.gz]: /dev/sda4/scripts.tar.xz
archive exists, overwrite? (y/n): y
archive created: scripts.tar.xz

The compression type to be used will depend on which extension is typed; tar has a nice option called --auto-compress. So, in the above example, typing ...tar.xz will use the LZMA compression algorithm. Just typing Enter on the archive name and the default archive.tar.gz will be used. The script also supports tab-completion for typing the archive name to help navigate folders and files.

compress can be found in my general-scripts repository.

Perl module installation

If doing Perl programming or if another package requires a Perl module, learning how to install one may become necessary. The recommended way to install a Perl module is through the distribution’s repositories, however, they can be installed manually with Perl.

Configure

Perl has its own repository where programmers make available their modules called the comprehensive Perl archive network, which is better known as the CPAN. Perl includes a built-in module that can download, build, and install from the network. For some distributions this module may already be built, however, it is probably a good idea for all to build it… to be sure it is set up correctly. Begin by starting the CPAN module shell so that it may be configured:

perl -MCPAN -e shell

A configuration message will appear… most users will be good with the automatic configuration it recommends. If additional configuring needs to be done later typing o conf init will re-run the configuration dialog. To leave the shell type exit.

Install

The first requirement most people will need to do is build and/or update the CPAN module. Modules can be installed with the built-in module in three ways: from the module shell, from the perl command, or from the CPAN module binary.

From the shell (which was entered in the configuration section), the following command will install a new module, or in this case, update the CPAN module:

install Bundle::CPAN

From the perl command:

perl -MCPAN -e 'install HTML::Template'

From the cpan module binary:

cpan Module::Name

Note: CPAN itself recommends using the cpanm module for installation. Modules will need to be reloaded after being updated: reload cpan.

Execute

Modules are sometimes executable binaries and if they are known to the shell can be executed like any other command. Some modules are support modules and can only be used for programming or by use of another module. Information of installed modules can be discovered with the command perldoc perllocal.

Uninstall

Module maintenance is typically unexpected after installation and the built-in CPAN module has no ability to be able to do so. If the cpanm module is installed it does have the ability with the --uninstall/-U option. It will display the files to be removed and prompt for approval before uninstalling.

System backup to DVD

The purpose of this article is to explain how to span large archives into multiple files. One would do this, for example, to store on numerous DVDs.

Rational

I had an occurrence where the only form of storage media I had were DVDs. It became necessary to create a full operating system backup and I was able to do so with the DVDs. (A system backup with DVDs is atypical because DVDs have dyes which have limited time usage.) The following explanation is for an Operating System multi-volume archive creation that can also be used on other large archives.

Operating System archive creation

From an Install CD I did my archive creation. Such an archive will nearly always need to be done from such an external operating system (a running operating system is always adding or editing files that necessitates using an external operating system).

My disk partitions I label according to the Operating System. The archive name I used begins with my hostname which is my computer model:

mkdir               /mnt/LABEL
mount /dev/disk/by-label/LABEL
cd                  /mnt/LABEL
archive_name=$HOSTNAME_$(date +%F)_DISTRO.tar.xz
tar cavf $archive_name .

Archive Multi-voluming

I can now divide the archive to multiple volumes. I do this so that they fit on a DVD. A few explanations first:

  • DVD storage capacity varies. For me I used a 4.7 Gigabyte DVD.
  • DVD storage capacity will likely need to be converted so a multi-volume file will fit on a DVD. DVD storage capacity is almost always calculated with metric prefixes (i.e. a base of 1000); however, typical computer numerology uses the binary prefixes a base of 1024. The metric 4.7 Gigabyte converted to binary is 4.37721 Gibibyte (GB to GiB conversion fraction: 1,000,000,000/[1024 x 1024 x 1024] = 0.9313226).
  • The UDF file system, typically used for DVD data storage, at the time of this writing was still experimental on Linux and I choose to use the ISO-9660 file system for reliability. However, this file system has a file size limit of four GiB. Since my writable DVDs had a capacity of 4.7 GB I had to split up the files to two per DVD. File system overhead also has to be factored in and I reduced 2.188608100 GiB to 2.188000 GiB.

There are two methods that typically do this:

split

I default to the split command because the open source philosophy “do one job and do it good”. It is easy to operate: -b/--bytes= is converted to Mebibytes and -d appends a numerical suffix. (Notice the period following to the second value.)

split -b 2240M -d HOSTNAME_DATE_DISTRO.tar.xz HOSTNAME_DATE_DISTRO.tar.xz.

tar

tar can also do it. Gibibytes is converted to Kibibytes (1024 bits). To _c_reate a _L_ength-limited _M_ulti-volume archive:

tar -cML 2284284 -f HOSTNAME_DATE_DISTRO.tar.xz.00 HOSTNAME_DATE_DISTRO.tar.xz

The first multi-volume file will be created and then the name of the next file will be requested:

Prepare volume #2 for HOSTNAME_DATE_DISTRO.tar.xz.00
# n HOSTNAME_DATE_DISTRO.tar.xz.01

If the save location is in another location the path will need to be entered on every new entry (e.g. n /mnt/Backup/HOSTNAME_DATE_DISTRO.tar.xz.02).

DVD writing

I believe growisofs is a good way to do this:

growisofs -Z /dev/dvd -rJ HOSTNAME_DATE_DISTRO.tar.xz.{00,01}

Operating System archive restore

Boot from the Install CD, then:

mkdir /mnt/LABEL
mount /dev/dvd /mnt/dvd
mount /dev/disk/by-label/LABEL /mnt/LABEL

File system creation I am assuming is already done. To join the multi-volume archives (if done by split and tar respectively):

cat      HOSTNAME_DATE_DISTRO.tar.xz.*  > HOSTNAME_DATE_DISTRO.tar.xz
tar -xMf HOSTNAME_DATE_DISTRO.tar.xz.00   HOSTNAME_DATE_DISTRO.tar.xz

Then the archive will just need decompressing:

tar xvf HOSTNAME_DATE_DISTRO.tar.xz -C /mnt/LABEL

External links

Patterns and textures

These patterns were originally for Nautilus, the GNOME file manager (now called “Files”), when it supported having images as backgrounds. They are still good for some other programs though so I touched them up a bit.

I designed these so readability was good with them which was the key factor. A background I feel is to help with the comfort-ability of the area. They are barely noticed and add grip to the area. An original example:

screenshot-jpasd-file-browser.png

And here is a view of them all:

example

Enjoy!

Firefox profile on a flash drive

firefox-tuning

I regularly use a computer at my workplace. This computer multiple people use so it is setup not to save Firefox’s settings. I use it enough, in specific ways, that I decided to find a way to use it with settings available.

I take a flash drive with me to be able to access various personal documents and programs. I had heard about web browsers being installed on flash drives; after I thought about it awhile, I realized all I would be required to do is put a profile on it.

The flash drive I have is formatted FAT32 to be able to use with Windows (my work computer) and I keep my flash drive organized similar to my Linux home directory for convenience. First I created a directory for the profile:

mkdir -p ~/.mozilla/firefox/profile/ANAME

Then I create the profile—from the command prompt this command will get the job done:

start firefox.exe -CreateProfile "MyName D:\.mozilla\firefox\profile\ANAME"

start-menu-example

(Or alternately I could have typed firefox.exe -P from the start menu and used the GUI version.)

I started the profile then to have the necessary files created. After it got done loading, I quit Firefox and I deleted the profile managers knowledge of the profile but not the profile itself (see example picture).

With the profile created all required effort left to be done is to instruct Firefox of the profile’s location. I put this in a batch script so that I can regularly use it:

@echo off

:: http://stackoverflow.com/a/15815897
:: http://stackoverflow.com/q/154075

set HOMEDRIVE=%cd:~0,2%

if exist C:\PATH\TO\firefox.exe start /b C:\PATH\TO\firefox.exe -profile %HOMEDRIVE%\.mozilla\firefox\profile\toddweed && exit

if exist C:\PATH\TO\firefox.exe start /b C:\PATH\TO\firefox.exe -profile %HOMEDRIVE%\.mozilla\firefox\profile\toddweed && exit

if exist C:\Program Files\Mozilla Firefox\firefox.exe (
  start /b C:\PATH\TO\firefox.exe -profile %HOMEDRIVE%\.mozilla\firefox\profile\toddweed
) else (
  echo Firefox executable not found.
)

lnk—forward thinking file linking

When I first used ln I tried using it before reading the documentation. I had assumed that linking was a basic enough operation to make the syntax ln [source-target] [linkname] all I needed to do. I learned though the common deployment of ln is otherwise. Since I created enough links, and because I felt the syntax should be basic, I created a script to get this behavior.

Besides a basic syntax that was logical to me, there are a few other reasons why I created the script. To know what they are, it helps to know the basics of linking.

Principalia linkathica

The default/non-optioned use of ln creates a hard link. A “hard link” is essentially just another name for an existing file. Because the hard link and its source (“target” in the documentation’s wording) share the same file system inode, they are almost indistinguishable (the inode contains all the information about a file).

Hard links are rarely used however. For several reasons its alternative a symbolic link is. While the ln default behavior does create a hard link, its existence is likely a inherited artifact—hard links came before symbolic links and program syntax had to be maintained to run as the users expected.

A symbolic link is more versatile than a hard link. It is sometimes referred to as a “symlink” or a “soft link” and it has some advantages. It can be:

  • readily used on directories
  • used across file system boundaries
  • created if the source/target doesn’t exist
  • color formatted with the ls command (and often is by default)

Further explanation of what a symbolic link is (as explained in the ln Info page, lightly paraphrased):

A symbolic link is a special type of file that refers to a different file by name. Most operations that are passed to the link file (opening, reading, writing…) are deferred by the kernel to operate on its target. Some operations (e.g. removing) work on the link file itself. The owner and group of a symlink have no effect on the file access of the target — they only have an effect on the removing of the symlink itself. On the GNU system, the file mode bits of a symlink have no significance and cannot be changed.

A symlink can be defined either with absolute or relative paths, the later being commonly used on removable media.

Examples:

cd $HOME
touch  file.txt
ln                 file.txt  file_hrdlink.txt
ln -s  /home/$USER/file.txt  /home/$USER/file_symlink-absolute.txt
ln -s     ../$USER/file.txt  file_symlink-relative.txt
ln -s     ../$USER/FILE.txt  file_symlink-relativebroken.txt

“Dance with the one that brung ya”

A basic syntax was what I wanted to be able to link by and why I created the script, additionally, a couple more benefits were able to be added:

  • symbolic links used by default as they are more flexible
  • absolute paths used for consistency and because they are usually more inductive to resolve
  • existence tests used on the source target and destination directory

Usage:

lnk [source-target] [directory-or-linkname] — a generic linker

Examples:

lnk can be found in my general-scripts repository.