GNU love for Windows

gwindows_logo

I wanted to be able to type in Windows with a familiar text editor. I downloaded a terminal text editor called gvim, which I think is a good text editor, and it was able to be installed portably—this last is necessary as the computer I use I am not able to install anything on. However, even with having that, I discovered that I wanted to be to use the GNU tools I had become familiar with… hence a dilemma.

There are several projects that provide GNU utilities on Windows… I have learned. I am not an expert in these mind you—I have tried only one—however, I’ve heard good things about them all. The following are all terminal emulators and they include the GNU utilities: Cygwin, Gow (GNU on Windows), and Git-bash.

I have only tried the later one. The reason for this is because I’ve had it already installed as it comes installed with the Git program that I use at times. I would have like to tried for the first two, however I’m not too picky and Git-bash has done me well enough; it has the basic utilities and is pretty much ready to go.

Having GNU utilities available is handy for me and saves me a bit of time because of familiarity; another bonus is time saved that would be required to learn the Windows command line. The tools run just like they do on Linux/Unix and can be used on the whole file system. Many of the tools are there: sed, awk, mkdir…. For example, I can type:

$ find /c/Windows/Web/ -type f -name "*Think*"
/c/Windows/Web/Wallpaper/Think/Think_Black.jpg
/c/Windows/Web/Wallpaper/Think/Think_Blue.jpg

By default the terminal emulator uses the already set Windows %HOMEPATH% variable for shell’s $HOME directory—this is usually C:\Users\USERNAME. I decided to use my flash drive as the $HOME directory where all my documents and settings could be kept. I had to create a batch script that defined the %HOMEPATH% and then have it start Git-bash:

@echo off

set HOMEDRIVE=%cd:~0,2%
set HOMEPATH=\

start /B %HOMEDRIVE%\Downloads\Git\git-bash.exe

01-git-bash-example

I then created a shortcut to flash drive root directory for quick access and to have a custom icon. I icon I choose was taken from the git-bash.exe file when I was asked for the icon location.

Git setup

An error was the first thing I had to fix… and it may just be for my particular version of Git; it complained to me when I tried to use it and I had to specify the certificate location. I did this by:

git config --system http.sslcainfo /d/Downloads/Git/mingw32/ssl/certs/ca-bundle.crt

After this I added ssh-agent to my ~/.bashrc launch it:

# SSH agent auto-launch
# 0 = agent running with key; 1 = agent running w/o key; 2 = agent not running
agent_run_state=$(ssh-add -l >| /dev/null 2>&1; echo $?)
if   [ $agent_run_state = 2 ]; then
  eval $(ssh-agent -s)
  ssh-add
elif [ $agent_run_state = 1 ]; then
  ssh-add
fi

And likewise I added to the bash_logout file ssh-agent -k as Windows would think that it was still running if I didn’t.

Now I’m working pretty good in Windows.

lnk—forward thinking file linking

When I first used ln I tried using it before reading the documentation. I had assumed that linking was a basic enough operation to make the syntax ln [source-target] [linkname] all I needed to do. I learned though the common deployment of ln is otherwise. Since I created enough links, and because I felt the syntax should be basic, I created a script to get this behavior.

Besides a basic syntax that was logical to me, there are a few other reasons why I created the script. To know what they are, it helps to know the basics of linking.

Principalia linkathica

The default/non-optioned use of ln creates a hard link. A “hard link” is essentially just another name for an existing file. Because the hard link and its source (“target” in the documentation’s wording) share the same file system inode, they are almost indistinguishable (the inode contains all the information about a file).

Hard links are rarely used however. For several reasons its alternative a symbolic link is. While the ln default behavior does create a hard link, its existence is likely a inherited artifact—hard links came before symbolic links and program syntax had to be maintained to run as the users expected.

A symbolic link is more versatile than a hard link. It is sometimes referred to as a “symlink” or a “soft link” and it has some advantages. It can be:

  • readily used on directories
  • used across file system boundaries
  • created if the source/target doesn’t exist
  • color formatted with the ls command (and often is by default)

Further explanation of what a symbolic link is (as explained in the ln Info page, lightly paraphrased):

A symbolic link is a special type of file that refers to a different file by name. Most operations that are passed to the link file (opening, reading, writing…) are deferred by the kernel to operate on its target. Some operations (e.g. removing) work on the link file itself. The owner and group of a symlink have no effect on the file access of the target — they only have an effect on the removing of the symlink itself. On the GNU system, the file mode bits of a symlink have no significance and cannot be changed.

A symlink can be defined either with absolute or relative paths, the later being commonly used on removable media.

Examples:

cd $HOME
touch  file.txt
ln                 file.txt  file_hrdlink.txt
ln -s  /home/$USER/file.txt  /home/$USER/file_symlink-absolute.txt
ln -s     ../$USER/file.txt  file_symlink-relative.txt
ln -s     ../$USER/FILE.txt  file_symlink-relativebroken.txt

“Dance with the one that brung ya”

A basic syntax was what I wanted to be able to link by and why I created the script, additionally, a couple more benefits were able to be added:

  • symbolic links used by default as they are more flexible
  • absolute paths used for consistency and because they are usually more inductive to resolve
  • existence tests used on the source target and destination directory

Usage:

lnk [source-target] [directory-or-linkname] — a generic linker

Examples:

lnk can be found in my general-scripts repository.

Keyboard outlines for key mapping

When I work with a program long enough I will at times like to write down the key mappings so I may refer to them later. I had done this enough that I decided I might as well make a keyboard outline to use for key mapping. I did four of them for the basic US-English models. The fonts Arial and Arial MT Rounded are required to see properly. Enjoy.

Compact layout

Full layout

Laptop layout

Mini layout

Example key mapping

Root directory residuals

root-directory-residualsThe only directory in my file system that I’d like to keep track of is my home directory. Here I keep my personal files and the a number of configurations that I love. I write notes of any configuration edits I make and record them here. However, besides the system configurations that I have edited, I also have a few more root directory configurations that I created. I wanted a way to keep track of them.

Getting down with the O C D

Previously I choose to reinstall about every six to twelve months. This allowed me to examine the install process to see if there were any new details about the operating system that I needed to learn. However, doing this procedure caused me to lose some good details I had put in the configurations. After I did this a few times, I began to do backups of them.

To keep the system running as expected, I learned I had to keep track of my configurations. The system configurations that I had edited originally I could keep track of by package updates (where I have to regularly merge the new versions to the old). However for the system configurations I created, I pretty much forgot them. These files occasionally I would rediscover when I had to do some troubleshooting. I came to the conclusion that if I wanted to track them, the best way to do so was to put them in a package`.

(I choose now to avoid installs when I can. These days, researching the install guide is enough to keep me up-to-date of operating system changes.)

Hunter and gatherer

I wasn’t always good at recording what configurations I created — I would test something out, or get excited when an experiment worked… Therefore, I had to search through the file system to re-track these files. This involved me making a list of all the files in the file system and comparing that to a list of files of the packages themselves. This sounds like a laborious process but isn’t terribly difficult and can be trimmed down greatly.

Because configurations are generally only in several directories less searching is required.

A file list of all packages, can be created with:

for p in $(pacman -Qq); do pacman -Qql $p; done | sed 's#/$##g' | sort -u -o pkgs-filelist.txt

A file list of the root directory, can be created with:

find / -not -path "/dev/*" -not -path "/home/*" -not -path "/media/*" -not -path "/mnt/*" -not -path "/proc/*" -not -path "/root/*" -not -path "/run/*" -not -path "/srv/*" -not -path "/sys/*" -not -path "/tmp/*" -not -path "/var/cache/*"  -not -path "/var/lib/pacman/*" -not -path "/var/lib/systemd/*" -not -path "/var/log/journal/*" -not -path "/var/tmp/*" | sed 's#/$##g' | sort -u -o root-filelist.txt

The differences can be viewed with:

vimdiff pkgs-filelist.txt root-filelist.txt

The package please… emmhh!

Creating a configuration package is the same as creating any other package. I put the configurations in the PKGBUILD directory and in it directed where to install them. In the future if an edit is required, I edit them there and rebuild it.

pkgbuild

Command line dictionary

command-line-dictionary

As a person who likes to write it has always been helpful for me to have a dictionary nearby. As a regular command line user to have a dictionary I could access from there was something I really wanted. I hadn’t predicted this would be much of a task, however, I found it an uphill battle.

What I felt a command line dictionary should offer:

1) a basic description that is accurate
2) the capability to be accessed offline
3) a formatting that is easy to read

Availability

In my original attempt I didn’t find any. I looked at a number of programs but most were inadequate in one way or another. I was baffled and I almost gave up looking. I did eventually find one but before that the two most promising programs were dictd and sdcv.

Dictd

Dictd is a protocol/software-framework for a networking dictionary, it contains both a server and a client. The idea is to have a server where numerous clients can connect to it. This would be useful for local network use or for something like an online dictionary group. However, it seems that the development has been quiet, and I had trouble installing several of its dictionaries… I could never get it to work.

The basic setup steps that are required to make it function are:

  1. install package and a dictionary for it
  2. start the dictd daemon (requires very little overhead) and check if the dictionaries are available (dict -I).
  3. look up a word definition using a particular dictionary (e.g. dict --database gcide)

sdcv

I used to use this program (Stardict console version) for years. It provided a basic, easy-to-use, unambiguous, definition. These days, however, the parent program StarDict is no longer in development. Additionaly, there were formatting problems that broke reading flow, and made it difficult to read.

Forest through the trees

I may not always get what I want, at other times, if I’m paying attention, I’ll find what I need. I discovered a program that while not a full-blown dictionary does pretty good. It technically might not even be a dictionary. From the man page:

wn - command line interface to the WordNet lexical database... it outputs synsets and relations to be displayed as formatted text.

In more human-speak: it details relationships between words. Its use as a thesaurus would be of a more direct comparison; however it can work for a dictionary as it does provide definitions and contextual examples. The definitions may be basic, but they are to the point. The only feature it does not provide that I use sometimes is word pronunciations.

wn lexical -over
...
The adj lexical has 2 senses (first 1 from tagged texts)
...
1. (2) lexical -- (of or relating to words; "lexical decision task")
2. lexical -- (of or relating to dictionaries) 

Creating good enough alone

The output of wn can be difficult to read: it jumbles a lot of information together, and only roughly organizes it. (FYI, in the above example I’ve filtered out a couple lines.) To help the reading of it in a smooth natural way, I’ve created a couple scripts to format the output. One script is called dict and the other is called thes.

dict

I’ve put them in a repository for any who are interested.

Two fine DAE scripts

fine-dae-scripts

Anybody that knows my command line habits, or me in general, knows that my memory could be better. It can be good when I need it to be, however, if I don’t have to remember a thing I’m writing it down. This is why I built my DAE scripts. DAE, which I pronounce as day, is an acronym for Digital Audio Extraction, also known as ripping audio CDs. The scripts are a wrapper for a command that has only a few options yet I have no way to remember the command options that I may not use again for awhile. Hence I created these basic scripts. They are straightforward scripts that just cover the essentials.

(These scripts are only basic wrappers, most of the work is done by the RipIt developer(s)… I thank them very much for their effort.)

How they look

There are two scripts. They are demonstrated here in use, as it is the best way to describe them.

daeme (pronounced like lame) is for MP3s:

daefe

daefe is for MP4s:

daefe

After these steps RipIt does a CDDB query from the Internet (if available) and allows tag editing if desired.

Settings

I bypassed adding a few settings in the script and rather allowed them to be specified in the RipIt configuration file as their values will likely remain the same:

faacopt=-s
dirtemplate="${artist} — ${album}"
playlist=0
eject=1

Audiobooks

The daefe script can also be used for audiobooks. The procedure encodes an entire CD to a file and writes a track/chapter index to another file. The chapter index file can be merged into the audiobook for an integrated audiobook, read ArchWiki:Audiobook for more details.

Download

Both scripts have error checking and I consider them reliable. I have put them in a repository for anyone interested.

That audio I do!

posticon

When I looked at my audio files recently, I realized that that I wanted them organized in a consistent way. As it were naming standards varied, some tags were missing, different encoding types were used… I decided I was gonna Feng Shui my way out of it.

The reorganization department

I decided there should be consistent naming and it should be condensed as much as possible, yet still being understandable.

For the directory format I use $artist—$album. For many people the format is $artist/$album, however, I came to the term that I would have to have about 100 CDs in my collection before the list would get too cumbersome to navigate.

I decided also to do one directory per album. Before I had directory names like Fleetwood Mac — The Very Best of Fleetwood Mac (disc 1), but I discovered it was much tidier to use a base title: Fleetwood Mac — The Very Best of Fleetwood Mac then prepend the disc number to the audio files:

? ls Fleetwood Mac — The Very Best of Fleetwood Mac/
...
1-15 Songbird.m4a
1-16 Big Love (Live, 1997).m4a
1-17 Storms.m4a
2-01 The Chain.m4a
2-02 Don't Stop.m4a
2-03 What Makes You Think You're the One.m4a
...

There is another program that organizes multiple CDs in this fashion and I like the thought behind it.

Just the FAACs

I had been creating high-quality MP3s (256kbps) and just accidentally stumbled into trying MP4s—and was delighted by the difference. Similar bitrates of audio sounded fractionally, but for me, appreciatively better. So I encoded all CDs to .m4a (MPEG-4 audio extension). (I will probably go to loss-free audio format in the future if the gods favor.)

By the way, I think (clearly subjectively) that FAAC (Free Advanced Audio Coding encoder) is great. It mentions in the manual that “it is not up to par with the currently best AAC encoders” but from my semi-proficient audio setup it did well.

I tested encoding with a FAAC at a setting of 320 (~256kbps) versus the iTunes setting of 256kbps. I did find iTunes better. Audiophiles looking for every detail might talk but for me difference could be left alone.

Here is a partially non-scientific (but should be fairly representative) graph of FAAC’s quality settings compared to kilobytes per second:

I can verify for the first and fourth values as I have tested a number of times. The second and third values I got from hydrogen audio. The fifth value is a projected value based from the other values.

A FAAC setting of 150 has been recommended for “casual, non-critical listening”; however, I use FAAC 320 for my tunes and 55 for voice. Though similar bitrates to the iTunes encoder created slightly less quality, the file size reflected too: iTunes 8.4MB, FAAC 7.6MB.)

Softer player detritus

Software audio players that I have used tended to put a some extraneous files in my audio folders. I’ve seen album cover art put here, lyrics, some unnecessary metadata, and hidden folders. I did myself a favor and deleted them out. Nothing should be here but the audio files. If an audio player insists on putting stuff here, I would recommend to their users to file a bug so the developer can remedy it. Some may argue that album playlists should be put here but these I believe belong to saving along with the configuration files.

Book ’em, Danno!

I discovered a nice feature call the audiobook format. Instead of 100 tracks splayed over a directory, or directories, from a audiobook CD, I can reduce this to a reasonable amount (say, one file per disc or even less if I choose). These book formats can also contain chapter indexes so navigating a file is just like navigating a disc. The process isn’t incredibly difficult and I documented it here. If the audio player supports them, it is a nice feature.