lnk—forward thinking file linking

When I first used ln I tried using it before reading the documentation. I had assumed that linking was a basic enough operation to make the syntax ln [source-target] [linkname] all I needed to do. I learned though the common deployment of ln is otherwise. Since I created enough links, and because I felt the syntax should be basic, I created a script to get this behavior.

Besides a basic syntax that was logical to me, there are a few other reasons why I created the script. To know what they are, it helps to know the basics of linking.

Principalia linkathica

The default/non-optioned use of ln creates a hard link. A “hard link” is essentially just another name for an existing file. Because the hard link and its source (“target” in the documentation’s wording) share the same file system inode, they are almost indistinguishable (the inode contains all the information about a file).

Hard links are rarely used however. For several reasons its alternative a symbolic link is. While the ln default behavior does create a hard link, its existence is likely a inherited artifact—hard links came before symbolic links and program syntax had to be maintained to run as the users expected.

A symbolic link is more versatile than a hard link. It is sometimes referred to as a “symlink” or a “soft link” and it has some advantages. It can be:

  • readily used on directories
  • used across file system boundaries
  • created if the source/target doesn’t exist
  • color formatted with the ls command (and often is by default)

Further explanation of what a symbolic link is (as explained in the ln Info page, lightly paraphrased):

A symbolic link is a special type of file that refers to a different file by name. Most operations that are passed to the link file (opening, reading, writing…) are deferred by the kernel to operate on its target. Some operations (e.g. removing) work on the link file itself. The owner and group of a symlink have no effect on the file access of the target — they only have an effect on the removing of the symlink itself. On the GNU system, the file mode bits of a symlink have no significance and cannot be changed.

A symlink can be defined either with absolute or relative paths, the later being commonly used on removable media.

Examples:

cd $HOME
touch  file.txt
ln                 file.txt  file_hrdlink.txt
ln -s  /home/$USER/file.txt  /home/$USER/file_symlink-absolute.txt
ln -s     ../$USER/file.txt  file_symlink-relative.txt
ln -s     ../$USER/FILE.txt  file_symlink-relativebroken.txt

“Dance with the one that brung ya”

A basic syntax was what I wanted to be able to link by and why I created the script, additionally, a couple more benefits were able to be added:

  • symbolic links used by default as they are more flexible
  • absolute paths used for consistency and because they are usually more inductive to resolve
  • existence tests used on the source target and destination directory

Usage:

lnk [source-target] [directory-or-linkname] — a generic linker

Examples:

lnk can be found in my general-scripts repository.

Keyboard outlines for key mapping

When I work with a program long enough I will at times like to write down the key mappings so I may refer to them later. I had done this enough that I decided I might as well make a keyboard outline to use for key mapping. I did four of them for the basic US-English models. The fonts Arial and Arial MT Rounded are required to see properly. Enjoy.

Compact layout

Full layout

Laptop layout

Mini layout

Example key mapping

Root directory residuals

root-directory-residualsThe only directory in my file system that I’d like to keep track of is my home directory. Here I keep my personal files and the a number of configurations that I love. I write notes of any configuration edits I make and record them here. However, besides the system configurations that I have edited, I also have a few more root directory configurations that I created. I wanted a way to keep track of them.

Getting down with the O C D

Previously I choose to reinstall about every six to twelve months. This allowed me to examine the install process to see if there were any new details about the operating system that I needed to learn. However, doing this procedure caused me to lose some good details I had put in the configurations. After I did this a few times, I began to do backups of them.

To keep the system running as expected, I learned I had to keep track of my configurations. The system configurations that I had edited originally I could keep track of by package updates (where I have to regularly merge the new versions to the old). However for the system configurations I created, I pretty much forgot them. These files occasionally I would rediscover when I had to do some troubleshooting. I came to the conclusion that if I wanted to track them, the best way to do so was to put them in a package`.

(I choose now to avoid installs when I can. These days, researching the install guide is enough to keep me up-to-date of operating system changes.)

Hunter and gatherer

I wasn’t always good at recording what configurations I created — I would test something out, or get excited when an experiment worked… Therefore, I had to search through the file system to re-track these files. This involved me making a list of all the files in the file system and comparing that to a list of files of the packages themselves. This sounds like a laborious process but isn’t terribly difficult and can be trimmed down greatly.

Because configurations are generally only in several directories less searching is required.

A file list of all packages, can be created with:

for p in $(pacman -Qq); do pacman -Qql $p; done | sed 's#/$##g' | sort -u -o pkgs-filelist.txt

A file list of the root directory, can be created with:

find / -not -path "/dev/*" -not -path "/home/*" -not -path "/media/*" -not -path "/mnt/*" -not -path "/proc/*" -not -path "/root/*" -not -path "/run/*" -not -path "/srv/*" -not -path "/sys/*" -not -path "/tmp/*" -not -path "/var/cache/*"  -not -path "/var/lib/pacman/*" -not -path "/var/lib/systemd/*" -not -path "/var/log/journal/*" -not -path "/var/tmp/*" | sed 's#/$##g' | sort -u -o root-filelist.txt

The differences can be viewed with:

vimdiff pkgs-filelist.txt root-filelist.txt

The package please… emmhh!

Creating a configuration package is the same as creating any other package. I put the configurations in the PKGBUILD directory and in it directed where to install them. In the future if an edit is required, I edit them there and rebuild it.

pkgbuild

Command line dictionary

command-line-dictionary

As a person who likes to write it has always been helpful for me to have a dictionary nearby. As a regular command line user to have a dictionary I could access from there was something I really wanted. I hadn’t predicted this would be much of a task, however, I found it an uphill battle.

What I felt a command line dictionary should offer:

1) a basic description that is accurate
2) the capability to be accessed offline
3) a formatting that is easy to read

Availability

In my original attempt I didn’t find any. I looked at a number of programs but most were inadequate in one way or another. I was baffled and I almost gave up looking. I did eventually find one but before that the two most promising programs were dictd and sdcv.

Dictd

Dictd is a protocol/software-framework for a networking dictionary, it contains both a server and a client. The idea is to have a server where numerous clients can connect to it. This would be useful for local network use or for something like an online dictionary group. However, it seems that the development has been quiet, and I had trouble installing several of its dictionaries… I could never get it to work.

The basic setup steps that are required to make it function are:

  1. install package and a dictionary for it
  2. start the dictd daemon (requires very little overhead) and check if the dictionaries are available (dict -I).
  3. look up a word definition using a particular dictionary (e.g. dict --database gcide)

sdcv

I used to use this program (Stardict console version) for years. It provided a basic, easy-to-use, unambiguous, definition. These days, however, the parent program StarDict is no longer in development. Additionaly, there were formatting problems that broke reading flow, and made it difficult to read.

Forest through the trees

I may not always get what I want, at other times, if I’m paying attention, I’ll find what I need. I discovered a program that while not a full-blown dictionary does pretty good. It technically might not even be a dictionary. From the man page:

wn - command line interface to the WordNet lexical database... it outputs synsets and relations to be displayed as formatted text.

In more human-speak: it details relationships between words. Its use as a thesaurus would be of a more direct comparison; however it can work for a dictionary as it does provide definitions and contextual examples. The definitions may be basic, but they are to the point. The only feature it does not provide that I use sometimes is word pronunciations.

wn lexical -over
...
The adj lexical has 2 senses (first 1 from tagged texts)
...
1. (2) lexical -- (of or relating to words; "lexical decision task")
2. lexical -- (of or relating to dictionaries) 

Creating good enough alone

The output of wn can be difficult to read: it jumbles a lot of information together, and only roughly organizes it. (FYI, in the above example I’ve filtered out a couple lines.) To help the reading of it in a smooth natural way, I’ve created a couple scripts to format the output. One script is called dict and the other is called thes.

dict

I’ve put them in a repository for any who are interested.

Two fine DAE scripts

fine-dae-scripts

Anybody that knows my command line habits, or me in general, knows that my memory could be better. It can be good when I need it to be, however, if I don’t have to remember a thing I’m writing it down. This is why I built my DAE scripts. DAE, which I pronounce as day, is an acronym for Digital Audio Extraction, also known as ripping audio CDs. The scripts are a wrapper for a command that has only a few options yet I have no way to remember the command options that I may not use again for awhile. Hence I created these basic scripts. They are straightforward scripts that just cover the essentials.

(These scripts are only basic wrappers, most of the work is done by the RipIt developer(s)… I thank them very much for their effort.)

How they look

There are two scripts. They are demonstrated here in use, as it is the best way to describe them.

daeme (pronounced like lame) is for MP3s:

daefe

daefe is for MP4s:

daefe

After these steps RipIt does a CDDB query from the Internet (if available) and allows tag editing if desired.

Settings

I bypassed adding a few settings in the script and rather allowed them to be specified in the RipIt configuration file as their values will likely remain the same:

faacopt=-s
dirtemplate="${artist} — ${album}"
playlist=0
eject=1

Audiobooks

The daefe script can also be used for audiobooks. The procedure encodes an entire CD to a file and writes a track/chapter index to another file. The chapter index file can be merged into the audiobook for an integrated audiobook, read ArchWiki:Audiobook for more details.

Download

Both scripts have error checking and I consider them reliable. I have put them in a repository for anyone interested.

That audio I do!

posticon

When I looked at my audio files recently, I realized that that I wanted them organized in a consistent way. As it were naming standards varied, some tags were missing, different encoding types were used… I decided I was gonna Feng Shui my way out of it.

The reorganization department

I decided there should be consistent naming and it should be condensed as much as possible, yet still being understandable.

For the directory format I use $artist—$album. For many people the format is $artist/$album, however, I came to the term that I would have to have about 100 CDs in my collection before the list would get too cumbersome to navigate.

I decided also to do one directory per album. Before I had directory names like Fleetwood Mac — The Very Best of Fleetwood Mac (disc 1), but I discovered it was much tidier to use a base title: Fleetwood Mac — The Very Best of Fleetwood Mac then prepend the disc number to the audio files:

? ls Fleetwood Mac — The Very Best of Fleetwood Mac/
...
1-15 Songbird.m4a
1-16 Big Love (Live, 1997).m4a
1-17 Storms.m4a
2-01 The Chain.m4a
2-02 Don't Stop.m4a
2-03 What Makes You Think You're the One.m4a
...

There is another program that organizes multiple CDs in this fashion and I like the thought behind it.

Just the FAACs

I had been creating high-quality MP3s (256kbps) and just accidentally stumbled into trying MP4s—and was delighted by the difference. Similar bitrates of audio sounded fractionally, but for me, appreciatively better. So I encoded all CDs to .m4a (MPEG-4 audio extension). (I will probably go to loss-free audio format in the future if the gods favor.)

By the way, I think (clearly subjectively) that FAAC (Free Advanced Audio Coding encoder) is great. It mentions in the manual that “it is not up to par with the currently best AAC encoders” but from my semi-proficient audio setup it did well.

I tested encoding with a FAAC at a setting of 320 (~256kbps) versus the iTunes setting of 256kbps. I did find iTunes better. Audiophiles looking for every detail might talk but for me difference could be left alone.

Here is a partially non-scientific (but should be fairly representative) graph of FAAC’s quality settings compared to kilobytes per second:

I can verify for the first and fourth values as I have tested a number of times. The second and third values I got from hydrogen audio. The fifth value is a projected value based from the other values.

A FAAC setting of 150 has been recommended for “casual, non-critical listening”; however, I use FAAC 320 for my tunes and 55 for voice. Though similar bitrates to the iTunes encoder created slightly less quality, the file size reflected too: iTunes 8.4MB, FAAC 7.6MB.)

Softer player detritus

Software audio players that I have used tended to put a some extraneous files in my audio folders. I’ve seen album cover art put here, lyrics, some unnecessary metadata, and hidden folders. I did myself a favor and deleted them out. Nothing should be here but the audio files. If an audio player insists on putting stuff here, I would recommend to their users to file a bug so the developer can remedy it. Some may argue that album playlists should be put here but these I believe belong to saving along with the configuration files.

Book ’em, Danno!

I discovered a nice feature call the audiobook format. Instead of 100 tracks splayed over a directory, or directories, from a audiobook CD, I can reduce this to a reasonable amount (say, one file per disc or even less if I choose). These book formats can also contain chapter indexes so navigating a file is just like navigating a disc. The process isn’t incredibly difficult and I documented it here. If the audio player supports them, it is a nice feature.

Home directory organization

organizing-my-home-directory

Being a Type A personality, I’ve probably thought more about this than many. I have a certain organizational style and formatting schema that I consistently think about.

Folder layouts

I keep all my home directories about the same as everyone:

# cd ~; ls -1p
Audio/
Desktop/
Development/
Documents/
Downloads/
Pictures/
Public/
Videos/

One exception to the above is the “Audio” directory which I use for multiple audio types:

# tree -L 1 -d Audio/
Audio/
├── Audiobooks
├── Music
├── Other
└── Podcasts

The other exceptions is I also hide the Templates directory (.Templates) as I don’t use it often.

Desktop

Even though I use GNOME I’ve enabled the Desktop—this is my workspace. If I can see them, I can remember them.

# ls -1 Desktop/ | head -n 3
arch-install.md
arch-linux-wiki-css-box_00.svg
command-line-dictionary.md
...

Development

I’ve come to be a big fan of VCSs. If I build something that others can use on their computer, I’ll create a VCS for it. I’ve put all of these VCS directories in their own directory:

# tree -L 1 -F -i --dirsfirst Development/
_vault/
archpkgs/
armrr/
arpa/
ar-utils/
daeme/
dotfiles/
general-scripts/
rback/
wordnet-dict/
xuserrun/

Documents

I put all the Documents in one directory whether I wrote them or obtained them:

# ls -1p Documents/
Blog/
Guides/
Receipts/
_vault/
car-maintenance-schedule.ods
recipes.md
wish-list.md
work-schedule-weekly-two-person.xls

Pictures

I do the same with Pictures as I do my Documents; whether I designed, photographed, or obtained them, I put them here:

# ls -1p Pictures/ | head -n 7
Camera/
Designs/
Nostalgia/
_vault/
aqua pr09studios.png
arch-pseudo3d_PJ.svg
architectural-intent.svg

Naming conventions

For the major folders, I use single words with the first letter uppercase. For files and other folders, I try to keep to the somewhat-traditional Linux method of naming my files as lowercase. For spaces in files, I generally use a hyphen (-) which I see used a lot these days though I think an underscore was originally used. Underscores I will use if there is a category I would like separated in the name (portrait-of-bach_etching.svg).

Audio files

Read this post to learn how I organize my audio files.

Vaults

When I come across a file that I won’t use anymore, is outdated, a misdirection, I create a folder called _vault and I place them in it. I then always have them around as I find that sometimes I like to.