Root directory residuals

root-directory-residualsThe only directory in my file system that I’d like to keep track of is my home directory. Here I keep my personal files and the a number of configurations that I love. I write notes of any configuration edits I make and record them here. However, besides the system configurations that I have edited, I also have a few more root directory configurations that I created. I wanted a way to keep track of them.

Getting down with the O C D

Previously I choose to reinstall about every six to twelve months. This allowed me to examine the install process to see if there were any new details about the operating system that I needed to learn. However, doing this procedure caused me to lose some good details I had put in the configurations. After I did this a few times, I began to do backups of them.

To keep the system running as expected, I learned I had to keep track of my configurations. The system configurations that I had edited originally I could keep track of by package updates (where I have to regularly merge the new versions to the old). However for the system configurations I created, I pretty much forgot them. These files occasionally I would rediscover when I had to do some troubleshooting. I came to the conclusion that if I wanted to track them, the best way to do so was to put them in a package`.

(I choose now to avoid installs when I can. These days, researching the install guide is enough to keep me up-to-date of operating system changes.)

Hunter and gatherer

I wasn’t always good at recording what configurations I created — I would test something out, or get excited when an experiment worked… Therefore, I had to search through the file system to re-track these files. This involved me making a list of all the files in the file system and comparing that to a list of files of the packages themselves. This sounds like a laborious process but isn’t terribly difficult and can be trimmed down greatly.

Because configurations are generally only in several directories less searching is required.

A file list of all packages, can be created with:

for p in $(pacman -Qq); do pacman -Qql $p; done | sed 's#/$##g' | sort -u -o pkgs-filelist.txt

A file list of the root directory, can be created with:

find / -not -path "/dev/*" -not -path "/home/*" -not -path "/media/*" -not -path "/mnt/*" -not -path "/proc/*" -not -path "/root/*" -not -path "/run/*" -not -path "/srv/*" -not -path "/sys/*" -not -path "/tmp/*" -not -path "/var/cache/*"  -not -path "/var/lib/pacman/*" -not -path "/var/lib/systemd/*" -not -path "/var/log/journal/*" -not -path "/var/tmp/*" | sed 's#/$##g' | sort -u -o root-filelist.txt

The differences can be viewed with:

vimdiff pkgs-filelist.txt root-filelist.txt

The package please… emmhh!

Creating a configuration package is the same as creating any other package. I put the configurations in the PKGBUILD directory and in it directed where to install them. In the future if an edit is required, I edit them there and rebuild it.

pkgbuild

Command line dictionary

command-line-dictionary

As a person who likes to write it has always been helpful for me to have a dictionary nearby. As a regular command line user to have a dictionary I could access from there was something I really wanted. I hadn’t predicted this would be much of a task, however, I found it an uphill battle.

What I felt a command line dictionary should offer:

1) a basic description that is accurate
2) the capability to be accessed offline
3) a formatting that is easy to read

Availability

In my original attempt I didn’t find any. I looked at a number of programs but most were inadequate in one way or another. I was baffled and I almost gave up looking. I did eventually find one but before that the two most promising programs were dictd and sdcv.

Dictd

Dictd is a protocol/software-framework for a networking dictionary, it contains both a server and a client. The idea is to have a server where numerous clients can connect to it. This would be useful for local network use or for something like an online dictionary group. However, it seems that the development has been quiet, and I had trouble installing several of its dictionaries… I could never get it to work.

The basic setup steps that are required to make it function are:

  1. install package and a dictionary for it
  2. start the dictd daemon (requires very little overhead) and check if the dictionaries are available (dict -I).
  3. look up a word definition using a particular dictionary (e.g. dict --database gcide)

sdcv

I used to use this program (Stardict console version) for years. It provided a basic, easy-to-use, unambiguous, definition. These days, however, the parent program StarDict is no longer in development. Additionaly, there were formatting problems that broke reading flow, and made it difficult to read.

Forest through the trees

I may not always get what I want, at other times, if I’m paying attention, I’ll find what I need. I discovered a program that while not a full-blown dictionary does pretty good. It technically might not even be a dictionary. From the man page:

wn - command line interface to the WordNet lexical database... it outputs synsets and relations to be displayed as formatted text.

In more human-speak: it details relationships between words. Its use as a thesaurus would be of a more direct comparison; however it can work for a dictionary as it does provide definitions and contextual examples. The definitions may be basic, but they are to the point. The only feature it does not provide that I use sometimes is word pronunciations.

wn lexical -over
...
The adj lexical has 2 senses (first 1 from tagged texts)
...
1. (2) lexical -- (of or relating to words; "lexical decision task")
2. lexical -- (of or relating to dictionaries) 

Creating good enough alone

The output of wn can be difficult to read: it jumbles a lot of information together, and only roughly organizes it. (FYI, in the above example I’ve filtered out a couple lines.) To help the reading of it in a smooth natural way, I’ve created a couple scripts to format the output. One script is called dict and the other is called thes.

dict

I’ve put them in a repository for any who are interested.

Two fine DAE scripts

fine-dae-scripts

Anybody that knows my command line habits, or me in general, knows that my memory could be better. It can be good when I need it to be, however, if I don’t have to remember a thing I’m writing it down. This is why I built my DAE scripts. DAE, which I pronounce as day, is an acronym for Digital Audio Extraction, also known as ripping audio CDs. The scripts are a wrapper for a command that has only a few options yet I have no way to remember the command options that I may not use again for awhile. Hence I created these basic scripts. They are straightforward scripts that just cover the essentials.

(These scripts are only basic wrappers, most of the work is done by the RipIt developer(s)… I thank them very much for their effort.)

How they look

There are two scripts. They are demonstrated here in use, as it is the best way to describe them.

daeme (pronounced like lame) is for MP3s:

daefe

daefe is for MP4s:

daefe

After these steps RipIt does a CDDB query from the Internet (if available) and allows tag editing if desired.

Settings

I bypassed adding a few settings in the script and rather allowed them to be specified in the RipIt configuration file as their values will likely remain the same:

faacopt=-s
dirtemplate="${artist} — ${album}"
playlist=0
eject=1

Audiobooks

The daefe script can also be used for audiobooks. The procedure encodes an entire CD to a file and writes a track/chapter index to another file. The chapter index file can be merged into the audiobook for an integrated audiobook, read ArchWiki:Audiobook for more details.

Download

Both scripts have error checking and I consider them reliable. I have put them in a repository for anyone interested.

That audio I do!

posticon

When I looked at my audio files recently, I realized that that I wanted them organized in a consistent way. As it were naming standards varied, some tags were missing, different encoding types were used… I decided I was gonna Feng Shui my way out of it.

The reorganization department

I decided there should be consistent naming and it should be condensed as much as possible, yet still being understandable.

For the directory format I use $artist—$album. For many people the format is $artist/$album, however, I came to the term that I would have to have about 100 CDs in my collection before the list would get too cumbersome to navigate.

I decided also to do one directory per album. Before I had directory names like Fleetwood Mac — The Very Best of Fleetwood Mac (disc 1), but I discovered it was much tidier to use a base title: Fleetwood Mac — The Very Best of Fleetwood Mac then prepend the disc number to the audio files:

? ls Fleetwood Mac — The Very Best of Fleetwood Mac/
...
1-15 Songbird.m4a
1-16 Big Love (Live, 1997).m4a
1-17 Storms.m4a
2-01 The Chain.m4a
2-02 Don't Stop.m4a
2-03 What Makes You Think You're the One.m4a
...

There is another program that organizes multiple CDs in this fashion and I like the thought behind it.

Just the FAACs

I had been creating high-quality MP3s (256kbps) and just accidentally stumbled into trying MP4s—and was delighted by the difference. Similar bitrates of audio sounded fractionally, but for me, appreciatively better. So I encoded all CDs to .m4a (MPEG-4 audio extension). (I will probably go to loss-free audio format in the future if the gods favor.)

By the way, I think (clearly subjectively) that FAAC (Free Advanced Audio Coding encoder) is great. It mentions in the manual that “it is not up to par with the currently best AAC encoders” but from my semi-proficient audio setup it did well.

I tested encoding with a FAAC at a setting of 320 (~256kbps) versus the iTunes setting of 256kbps. I did find iTunes better. Audiophiles looking for every detail might talk but for me difference could be left alone.

Here is a partially non-scientific (but should be fairly representative) graph of FAAC’s quality settings compared to kilobytes per second:

I can verify for the first and fourth values as I have tested a number of times. The second and third values I got from hydrogen audio. The fifth value is a projected value based from the other values.

A FAAC setting of 150 has been recommended for “casual, non-critical listening”; however, I use FAAC 320 for my tunes and 55 for voice. Though similar bitrates to the iTunes encoder created slightly less quality, the file size reflected too: iTunes 8.4MB, FAAC 7.6MB.)

Softer player detritus

Software audio players that I have used tended to put a some extraneous files in my audio folders. I’ve seen album cover art put here, lyrics, some unnecessary metadata, and hidden folders. I did myself a favor and deleted them out. Nothing should be here but the audio files. If an audio player insists on putting stuff here, I would recommend to their users to file a bug so the developer can remedy it. Some may argue that album playlists should be put here but these I believe belong to saving along with the configuration files.

Book ’em, Danno!

I discovered a nice feature call the audiobook format. Instead of 100 tracks splayed over a directory, or directories, from a audiobook CD, I can reduce this to a reasonable amount (say, one file per disc or even less if I choose). These book formats can also contain chapter indexes so navigating a file is just like navigating a disc. The process isn’t incredibly difficult and I documented it here. If the audio player supports them, it is a nice feature.

Home directory organization

organizing-my-home-directory

Being a Type A personality, I’ve probably thought more about this than many. I have a certain organizational style and formatting schema that I consistently think about.

Folder layouts

I keep all my home directories about the same as everyone:

# cd ~; ls -1p
Audio/
Desktop/
Development/
Documents/
Downloads/
Pictures/
Public/
Videos/

One exception to the above is the “Audio” directory which I use for multiple audio types:

# tree -L 1 -d Audio/
Audio/
├── Audiobooks
├── Music
├── Other
└── Podcasts

The other exceptions is I also hide the Templates directory (.Templates) as I don’t use it often.

Desktop

Even though I use GNOME I’ve enabled the Desktop—this is my workspace. If I can see them, I can remember them.

# ls -1 Desktop/ | head -n 3
arch-install.md
arch-linux-wiki-css-box_00.svg
command-line-dictionary.md
...

Development

I’ve come to be a big fan of VCSs. If I build something that others can use on their computer, I’ll create a VCS for it. I’ve put all of these VCS directories in their own directory:

# tree -L 1 -F -i --dirsfirst Development/
_vault/
archpkgs/
armrr/
arpa/
ar-utils/
daeme/
dotfiles/
general-scripts/
rback/
wordnet-dict/
xuserrun/

Documents

I put all the Documents in one directory whether I wrote them or obtained them:

# ls -1p Documents/
Blog/
Guides/
Receipts/
_vault/
car-maintenance-schedule.ods
recipes.md
wish-list.md
work-schedule-weekly-two-person.xls

Pictures

I do the same with Pictures as I do my Documents; whether I designed, photographed, or obtained them, I put them here:

# ls -1p Pictures/ | head -n 7
Camera/
Designs/
Nostalgia/
_vault/
aqua pr09studios.png
arch-pseudo3d_PJ.svg
architectural-intent.svg

Naming conventions

For the major folders, I use single words with the first letter uppercase. For files and other folders, I try to keep to the somewhat-traditional Linux method of naming my files as lowercase. For spaces in files, I generally use a hyphen (-) which I see used a lot these days though I think an underscore was originally used. Underscores I will use if there is a category I would like separated in the name (portrait-of-bach_etching.svg).

Audio files

Read this post to learn how I organize my audio files.

Vaults

When I come across a file that I won’t use anymore, is outdated, a misdirection, I create a folder called _vault and I place them in it. I then always have them around as I find that sometimes I like to.

Home directory regenesis

When some unexpected event occurs on my computer, I may begin by troubleshooting in the home directory. Lately, I was having several program loadings that were taking a good deal of time that explanation thereof was baffling—a resolution I was unfortunately not able to find an answer for. The realistic solution for me was to create a new user and copy the trusted files over to a new home folder. It turned out to be a fairly easy thing to do.

The quick and the darned

In the past methodically I would clean out my home directory. I did this every year or twice a year. I did this because usually a quirk developed here or there but also because I’m a type A personality. Why quirks occur can be of one of several reasons but it might be related to a program’s configuration files: like with the addition of a new feature to a program, or with an older setting interfering with a new feature, databases getting too large…

I have much respect for my configurations and I do my best to keep them. The methodology I have learned of how to interact with my programs I respect and try to keep around and develop. However, at times, configuration renewal may become necessary. I do this only when I have to.

All work and no play is OK

Business if more important when I come down to it. Going through all the files, is no doubt, a laborious process, however, the result is worth it when everything is again running correctly.

First, I create a new user:

# useradd --gid users --groups games,wheel --shell "/bin/bash" USERNAME 
# install --directory --owner=USERNAME --group=users --mode=700 /home/USERNAME
# passwd USERNAME

I then put all the directories and trusted configurations in a list:

.local/share/applications/
.local/share/gedit/styles/autumnal.xml
Audio
Desktop
Development
Documents
Downloads
Pictures
Public
Videos

Then transfer them using rsync:

rsync --archive --files-from=include.txt --exclude-from=exclude.txt /home/USERNAME-OLD/ /home/USERNAME-NEW/

Since both users are in the same initial group (users), I just need to change user ownership from the old own to the new and then I’m done:

find /home/USERNAME-NEW/ -user USERNAME-OLD -exec chown USERNAME-NEW {} \;

This process took me about an hour.

Vim colorscheme customization

Any installed Vim colorscheme has the ability to be customized. They can be tested temporarily or written to a supplemental file that will leave the original colorscheme file intact.

Testing

To customize a colorscheme value, Vim has on-the-fly colorscheme alteration support. To get an overview of what all the values look like:

:highlight

To see a specific value (tip: have wildmenu enabled for tab-completion support):

:highlight CursorLine

02_vim-cs-cursorline

To customized a value:

:highlight CursorLine ctermbg=255

Tips:

  • hi = highlight
  • If working from the terminal, it is useful to know what colors are available. A number of scripts can be found; I created one called termcolors.
  • A test can highlight the current groups: :so $VIMRUNTIME/syntax/hitest.vim

After directory

Vim uses the system directory $VIMRUNTIME/after/ and the local directory $HOME/after/ to supplement or overrule to the default settings. For editing colorschemes there will need to be a sub-directory called colors. To install it locally:

mkdir -p $HOME/.vim/after/colors

For neither, however, does Vim yet support colorscheme customizations and a plug-in will need to be installed: AfterColors. I would recommend using a plug-in manager like Vundle or Pathogen to install it.

Customization file

It may be a good idea to begin customizing by reading the colorscheme author’s notes. Authors will sometimes explain their design philosophy in the colorscheme file. These files are located system-wide in /usr/share/vim/vimfiles/colors/ or locally in ~/.vim/colors.

Customization values are put in a file that matches the colorscheme’s file name:

touch ~/.vim/after/colors/desert.vim

The values placed in the files are the same that can be done while testing the customizations:

" Vim colorscheme customizations: desert
highlight CursorLine ctermbg=255
highlight Visual     ctermbg=45

Save the file and reload the colorscheme to see the edits:

:colorscheme desert

… or type colo for the abbreviated version.