Command Line Calculator

I can usually type faster in the terminal than doing mouse click on a gui calculator, so I created this scipt to be able to do it quickly from the terminal. There are alot of command line calculators out there so use the one you are comfortable with but I like using bc because of the syntax. For example, you can type:

calc "6/(3*10)"

or something more complex:

calc "8^2*(10/2+(13.2032565*2030)/.349548)" 100

100 is optional, it will specify how many decimals you want to carry it out to (the default is 4).

	#!/bin/sh
	# Command line calculator
	# Display usage if no parameters given
	if [ -z "$@" ]; then
	echo " ${0##*/} <input> <*decimals> - command line calculator (-h for examples)"
	exit
	fi
	# Decimal to be carried out to (uses four unless value is given)
	if [ -z "$2" ]; then
	decimals=4; else
	decimals=$2
	fi
	case $1 in
	-h | --help ) echo " Requires quotes. Examples:"
	echo ' calc "5*((2^3*2)/1.352)"'
	echo ' calc "p=10; n=9; p*n+10"' ;;
	* ) echo "scale=$decimals;$1" | bc
	esac

view raw calc hosted with ❤ by GitHub

For the truly paranoid

I’ve been reinstalling my system as of late (been way too along a comin’) and I realized that I hadn’t set up a firewall yet. This, in turn, had me think how many ports were open. I was up too late and probably had too many cokes by then. I had given myself a dead simple root password so that I could finish the install and began getting that tightening, turning, wretching in the belly feeling. I couldn’t help thinking that, “This could be the time that some random joe comes along and finds a nice open gate”. Doesn’t make much sense now, but decided then to build a script that toggles a 20 character random password to relieved my paranoia. Here it is for anyone who can find use of it. Oh, and I did get my install done.

#!/bin/bash
# randompass - toggle between random and known passwords for users

# User passwords to protect
users=(root todd akau)

# Program name from it's filename
prog=${0##*/}

# Text color variables
txtund=$(tput sgr 0 1)          # Underline
txtbld=$(tput bold)             # Bold
bldred=${txtbld}$(tput setaf 1) #  red
bldblu=${txtbld}$(tput setaf 4) #  blue
bldwht=${txtbld}$(tput setaf 7) #  white
txtrst=$(tput sgr0)             # Reset
info=${bldwht}*${txtrst}        #
pass=${bldblu}*${txtrst}
warn=${bldred}!${txtrst}

# Check if users exist, if they don't remove from the users array
list="${users[@]/%/|}"    # Puts array to list, add pipe after each user
users=($(grep -Eo "^(${list// })\>" /etc/shadow)) # strip spaces, end of word

# Password generation
passgen=$(< /dev/urandom tr -dc A-Za-z0-9/.$ | head -c20 | xargs | cat)

# Variables for current passwords
for user in ${users[@]}; do
  eval "curpw$user=\$(grep \$user /etc/shadow | awk -F : '{print \$2}')"
done

# Save original passwords (first run)
for save in ${users[@]}; do
  if [ ! -f /root/pass$save ]; then
    grep $save /etc/shadow | awk -F : '{ print $2 }' > /root/pass$save
    echo "$pass Saved ${txtund}$save${txtrst} password"
  fi
done

case $1 in
  h ) echo " $prog <*u>- toggle random and known passwords. u - update known"
      ;;
  u ) echo "$warn Be sure no random passwords are set before updating passwords!"
      echo -n "Update known passwords file(s)? "
      read update
      if [[ $update == [Yy] ]]; then
        for known in ${users[@]}; do
          grep $known /etc/shadow | awk -F : '{ print $2 }' > /root/pass$known
          echo "$pass Updated ${txtund}$known${txtrst} password"
        done
        else
        echo " Passwords not updated"
        exit
      fi
      ;;
  * ) if [[ "$curpwroot" == "$(cat /root/passroot)" ]]; then
        for u in ${users[@]}; do
          usermod -p $passgen $u
          echo "$pass Generated password for ${txtund}$u${txtrst}."
        done
        else
        for u in ${users[@]}; do
          usermod -p $(cat /root/pass$u) $u
          echo "$pass Restored password for ${txtund}$u${txtrst}."
        done
      fi
      ;;
esac

HTML Entities from the Command Line

While doing HTML work I tend to do my work with text editors. For this, I use Arachnophilia a Java HTML editor with easy, editable, customizable tags (Review here).

Arachnophilia has support to convert characters to HTML entities but isn’t easy to get to (HTML > More Functions > Char to Entity. There are various web sites that do but if willing to use the terminal they can be quickly gotten there as well. Thanks to script by Darren this can be done easily. It requires script Perls’ HTML::Entities module to do so (for help installing Perl modules look at this page). You’ll probably need redirect the script to point to the Perl program proper:

whereis perl

More than likely its in /usr/bin/perl. After fixing that run the script. This will put you in a sub-shell that you can copy and paste characters to be encoded:

You can also convert a whole file. This will print to standard output (terminal text):

htmlentities filename

Or convert a file by doing:

htmlentities  < file > convertedfile

theatertime- hold power-saving to get through a flick

Getting down to watching you favorite movie on your computer? Start the movie, sit down, grab your snack and ten or so minutes later the screen goes blank. This happens in Linux because the desktop has built-in defaults for display power management (DPMS) and screensaving. Timeout settings can vary from distribution to distribution but they all got them. Here’s a basic script that can toggle DPMS and screensaving on and off.

Xorg Server Settings

You can set the values of blank, standby, suspend, and off in the the xorg server configuration file. The defaults are: 15, 20, 30, and 40 minutes. Personally I like to set these to better match how I use my computer:

Section "Monitor"
  Identifier  "Monitor0"
  Option      "DPMS"    "true"  # display power management on (true/false)
EndSection

Section "ServerFlags"
  Option "BlankTime"    "13"    # LED still on, no + (0 disables)
  Option "StandbyTime"  "15"    # turns off LED
  Option "SuspendTime"  "0"     # turns off LED, and most power
  Option "OffTime"      "50"    # turns off all power
EndSection

BlankTime is just a cheap screensaver and only real use for me is to tell me that I forgot to disable dpms while watching a movie. Doing this saves me a few seconds that StandbyTime requires to turn on the display again. SuspendTime and StandbyTime are nearly the same thing so I don’t bother setting SuspendTime.

Movietime

Here’s movietime. Movietime should work with just about any type of desktop environment (at least any system with dbus installed which really all of them do). If you aren’t familiar with having your own scripts and how to run them, take a look at this page).

#!/bin/bash
# movietime - disables power savings to watch movies.

# Movietime options
#  Resume time - resume normal display pm and suspend after set time.
# 0 = disabled, time in minutes
resumetime=0
if [ $resumetime = 0 ]; then 
  resumetime=1440 # Re-enable resume after a full day
fi

# Check that values for 'resumetime' are numbers
if [ $(echo $resumetime | sed 's/^[-+0-9][0-9]*//' | wc -c) != 1 ]; then
  echo "$warn variable 'resumetime' is not a number.  Exiting."
  exit
fi

# Name of suspend script
tmploc="/tmp"
suspinhscript="$tmploc"/"movietime-suspend-inhibit"

# Program name from it's filename.
prog=${0##*/}

# Text color variables
txtund=$(tput sgr 0 1)          # Underline
txtbld=$(tput bold)             # Bold
bldblu=${txtbld}$(tput setaf 4) #  blue
bldwht=${txtbld}$(tput setaf 7) #  white
bldred=${txtbld}$(tput setaf 1) #  red
txtrst=$(tput sgr0)             # Reset
info=${bldwht}*${txtrst}        # Feedback
pass=${bldblu}*${txtrst}
warn=${bldred}*${txtrst}

# Check that Xorg server is running
if [[ -z $(ps aux | grep /usr/bin/X) ]]; then
  echo "$warn The Xorg server is not running."
  exit
fi

# Check if user is regular user
if [ $(whoami) == "root" ]; then
  echo "$warn You are the root user, must be a regular user."
  exit
fi

# Current DPMS times (in minutes)
dispdpms=$(xset -q | grep "DPMS is" | awk '{ printf $3 }') # Enab. or Disb.
dispstand=$(xset -q | grep "^  Standby: " | awk '{ printf $2/60 }')
dispsusp=$(xset -q | grep "^  Standby: " | awk '{ printf $4/60 }')
dispoff=$(xset -q | grep "^  Standby: " | awk '{ printf $6/60 }')
dispblank=$(xset -q | grep "^  timeout:  " | awk '{ printf $2/60 }')

# Resume time in hours
resumetimehr=$(echo "scale=1;${resumetime}/60" | bc)

# Display help
case $1 in
  -h | --help | h | help )
    # Help message.
    echo
    echo "  $prog disables screen blanking and screensaver to allow viewing a video.  Running the program again will enable them.  If the 'resumetime' variable is set after that time $prog will resume normal powersaving values." | fmt -c -w 76
    echo

    # Display current values of power management and movietime.
    suspinhtest=$(ps aux | grep -v grep | grep $suspinhscript)
    suspinhval=$([ -n "$suspinhtest" ] && echo "Disabled" || echo "Desktop settings")
    
    # DPMS disabled information
    echo "   ${txtbld}Current settings ${txtrst}(in minutes, 0 = disabled):"
    if [[ "$dispdpms" == "Disabled" ]]; then
      echo "   DPMS:         $dispdpms"
      echo "   Suspend:      $suspinhval"
    fi

    # DPMS enabled information
    if [[ "$dispdpms" == "Enabled" ]]; then
      echo "   DPMS:         $dispdpms"
      echo "   DPMS times:   Blank: ${dispblank}; Standby: ${dispstand}; Suspend: ${dispsusp}; Offtime: ${dispoff}"
      echo "   Suspend:      $suspinhval"
    fi
    echo
    echo "   ${txtbld}$prog settings${txtrst}:"
    echo "   Resume after: $resumetimehr hours"
    echo
    exit
    ;;
  [a-g,i-z,A-G,I-Z,0-9,-]* )
    echo " Use '-h' for help"
    exit
    ;;
esac

# Suspend inhibit script (must be run as seperate process)
suspinhibit () {
    echo '#!/bin/bash
    for time in $(seq 1 '$resumetime'); do
    # Simulate user activity every minute
    dbus-send --print-reply --type=method_call --dest=org.freedesktop.ScreenSaver /ScreenSaver org.freedesktop.ScreenSaver.SimulateUserActivity
    sleep 60
    done'
}

# Toggle powersaving
if [[ "$dispdpms" == "Enabled" ]] && [ -z "$suspinhtest" ]; then
  # Disable blanking, screen power saving
  xset s off; xset -dpms
  # Create script in tmp
  suspinhibit > "$suspinhscript"
  # Make script executable
  chmod u+x "$suspinhscript"
  # Run script
  nohup "$suspinhscript" &> /dev/null &
  echo "$pass $prog started, powersaving disabled."
else
  # Enable blanking, screen power saving
  xset s on; xset +dpms
  # Kill script
  if [ -n $suspinhtest ]; then
    echo "$info $prog stopped, powersaving enabled."
    kill -s 9 $(pgrep movietime-susp) &> /dev/null
  fi
fi

Turn off all cellphones and enjoy the show!

Week of bash scripts – rps and commentstrip

These two scripts will respectively: find if a program is running, and strip-comments from text files. The first is useful if you need to see if the program is running or if you need to kill the process with it’s id, comment strip is a good tool to use if posting configurations on forums as often developers or advanced users already know what the settings actually do.

rps

aspire ~/.scripts:
rps geany
todd      1827  0.1  0.3 184576 28616 ?        S    May31   1:05 geany

	#!/bin/bash
	# Running program search
	ps aux | grep --color=always -i "$@" | grep -v grep | grep -v "$0"

view raw rps hosted with ❤ by GitHub

commentstrip

Commentstrip will display the output to the terminal. If you got xclip installed the ‘c’ option can be used to copy the output to the clipboard.

	#!/bin/bash
	# Output file without comments or blanklines
	# Display usage if no parameters given
	if [ -z $1 ]; then
	echo " ${0##*/} <*c> <filename> - print file w/o comments/blanklines - (c)lipboard"
	exit 1
	fi
	case $1 in
	# Copy output to xorg server clipboard
	c ) shift
	# Check if selection exists
	if [ ! -f "$@" ]; then
	echo " Selection \""$@"\" does not exist." && exit
	fi
	# Exit if root (root doesn't have access to user xorg server)
	if [[ `whoami` == root ]]; then
	echo " Copying to clipboard cannot be user root" && exit
	else
	grep -vh '^[[:space:]]*\(#\|$\)' "$@" | xclip -selection c
	echo " Comments stripped from file and copied to xorg server clipboard"
	fi
	;;
	# Print output to terminal
	* ) if [ ! -f "$@" ]; then
	echo " Selection \""$@"\" does not exist." && exit
	fi
	grep -vh '^[[:space:]]*\(#\|$\)' "$@" ;;
	esac

view raw cmtstrip hosted with ❤ by GitHub

The final day of week of bash scripts… phew! I’d like to thank everyone that posted comments, and to those that stopped by this week.

Week of bash scripts – grok and cdf

These two scripts are two different find commands. The first (grok) will list all files in a directory recursively that contain a matched string; the second will locate a file/folder and the change to it’s directory. Neither of these are mine (though slightly edited), I’ve gotten them from the Arch forums where they have a great thread called Post your handy self made command line utilities.

Grok

This one is by rebugger and I call it grok. Syntax is:

grok <string> <*location>

If no location is given it uses the current directory.

aspire ~:
grok 127.0.0.1 /etc/
 Searching...
/etc/dnsmasq.conf
/etc/dnsmasq.conf.pacorig
/etc/hosts
/etc/NetworkManager/dispatcher.d/localhost-prepend
/etc/ntp.conf
/etc/resolv.conf
/etc/security/access.conf
/etc/xinetd.d/servers
/etc/xinetd.d/services

	#!/bin/bash
	# Search file(s) for keyword
	# Display usage if no parameters given
	if [[ -z "$@" ]]; then
	echo -e " ${0##*/} <string> <file/path> - search recursively for keyword in files"
	exit
	fi
	if [ -z "$2" ]; then
	loc=$(pwd)
	else
	loc="$2"
	fi
	echo " Searching..."
	grep -ilr "$@" "$loc" 2>&1 | grep -v "No such" | grep -v "Permission denied" | sed "s:^\.::g" | sed "s:$(pwd)::g" | sort | uniq

view raw grok hosted with ❤ by GitHub

cdl

This one is by segoe that uses locate to find a file and that cd’s to the first match found.

aspire ~/.scripts:
cdl demo/PKG
aspire ~/.arch/pkgbuilds/amnesia-demo:

This one put in your ~/.bashrc:

cdf () { cd "$(dirname "$(locate -i "$*" | head -n 1)")" ; } # locate then cd

Week of bash scripts – newx and bgcmd

Here are two scripts: one, that helps improve gaming performance, and the other to free up the terminal. I’ve talked about the later before, but I got a new trick up my sleeve. However, before I go any further, I’d like to point out how to use bash scripts so that they are easily accessible, yet (at the same time) out of the way.

Organization

I like to keep my bash scripts out of plain view. Some people like to put their bash scripts in a folder called scripts in their home folder. This works good but keeping the home folder just for documents can help reduce clutter:

Think about putting your scripts in a hidden directory in your home folder. For example, I use a folder called ~/.bin. To make the scripts executable anywhere, create a path to them in your ~/.bashrc:

export PATH="/home/gen2ly/.bin:$PATH

To quickly enter the script directory, you might want to create an alias to it:

alias cdb="cd /home/gen2ly/.bin && ls -h"

Now, reload the bash environment:

source ~/.bashrc

To get to your script directory you can now just do:

After you create a script or download one, it will need to be made executable. You can do this by:

chmod +x script-name

NewX

A basic script but useful for gamers that don’t have the most powerful graphic cards. Compositing can zap game performance pretty thoroughly. Rather than digging through menus and disable compositing this command will just start a new xorg server:

#!/bin/bash

DISPLAY=:1.0

xinit $* -- :1

By typing newx a new xorg server will be opened on the eighth virtual console (Ctl+Alt+F8). This will also open a terminal where commands can be entered. Typing exit will exit the new xorg server and return you to your original. Alternatively you can type newx urbanterror and urbanterror will be loaded.

Bgcmd

bgcmd will background a program so that it doesn’t overtake the terminal. I’ve written about this before but I’ve discovered how to add bash-completion to it. I’ve updated the page to reflect such:

Background a Process

Enjoy!

Restore settings of Firefox on trouble

Update: 09-29-11 – Using script to automate process, see end of post.

When people have a issue with Firefox I’ve seen many people will resort to deleting their old profile (or folder) and creating a new one. This works but doing this will get rid of any passwords, history, bookmarks… therein. Having used Firefox quite a bit creating a new profile from time to time is a good idea anyhow as cruft, bad extensions, … can slow down browsing.

Manually

Copying the Firefox configs can be done by:

cd ~/.mozilla/firefox/

Backup the old profile and profile list:

mv xxxxxxxx.default{,.bck}
mv profiles.ini{,.bck}

Create a new profile:

firefox -CreateProfile <profilename>

This command will return the name of the new folder. Copy the basic settings to the new profile:

cd *.default.bck
cp places.sqlite key3.db cookies.sqlite mimeTypes.rdf formhistory.sqlite signons.sqlite permissions.sqlite webappsstore.sqlite persdict.dat content-prefs.sqlite ../*.<profilename>

This will transfer the bookmarks, browsing history, form entries, passwords, personal dictonary changes, and page zooms. There might be a couple other things wanted to add (possibly your firefox preferences), take a look at Transferring data to a new profile for more information.

	#!/bin/bash
	# Create new profile and transfer settings (fixes certain
	# problems when there are errors in Firefox settings).
	ffsettingsdir=~/.mozilla/firefox/
	# Setting files to restore (files to copy):
	# (http://kb.mozillazine.org/Transferring_data_to_a_new_profile_-_Firefox)
	files="content-prefs.sqlite
	cookies.sqlite
	formhistory.sqlite
	key3.db
	mimeTypes.rdf
	permissions.sqlite
	places.sqlite
	searchplugins
	signons.sqlite
	webappsstore.sqlite"
	# Change to Firefox settings directory
	if [ -d $ffsettingsdir ]
	then
	cd "$ffsettingsdir"
	else
	echo " Firefox settings directory "$ffsettingsdir" does not exist"
	exit
	fi
	# Select current profile directory
	printf "Select current Firefox profile settings directory:\n"
	select dir_current in */
	do
	test -n "$dir_current" && break
	echo " Select 1, 2, ..."
	done
	# Check that all listed setting files exist
	for f in $files
	do
	if [ ! -e "$dir_current""$f" ]
	then
	echo "File "$f" does not exist, exiting."
	exit
	fi
	done
	# Rename current profile directory
	mv "${dir_current%/}"{,.bck}
	dir_old="${dir_current%/}".bck
	echo "Renamed old profile directory to: "$dir_old""
	# Rename profile list file
	if [ -f profiles.ini ]
	then
	mv -f profiles.ini{,.bck}
	fi
	# Create new profile
	read -p "Name of new profile: " nw_prfl_nm
	firefox -CreateProfile "$nw_prfl_nm"
	# Select new profile directory
	printf "Select NEW Firefox profile settings directory:\n"
	select dir_new in */
	do
	test -n "$dir_new" && break
	echo " select 1 , 2 ..."
	done
	# Copy settings (files) from old profile to new profile
	cd "$dir_old"
	for f in $files
	do
	cp -a "$f" "$ffsettingsdir"/"$dir_new"/
	done

view raw ff-restore hosted with ❤ by GitHub

Personal LiveUSB

If you ever have an emergency and need a rescue disk to recover your Linux install, or maybe you just want to brag to your friends there’s some good LiveCD/USB’s out there and many distro’s now make LiveUSB install images, but it is also possible to create your own customizable LiveUSB. Hey, if you’re willing to put the time in, you can have a portable Linux in your pocket.

There’s alot of articles about creating your own custom CD/LiveUSB but many of them seemed dramatic involving messing with things like syslinux… Plus many of these create a fixed image, meaning that once it’s on your USB it can’t be changed. But having a customizable Linux on a USB flashdrive isn’t that difficult – just install Linux to the USB drive.

Partition the USB Drive

The first thing you’ll need is at least a 2GB flash drive. Anything less and you better plan a real basic install. First thing you might like to do is partition the flash drive. This isn’t necessary but I was pretty sure I wouldn’t need the 4GB for what I needed so I partitioned the flash drive to have a 1GB FAT32 partition first (so that Vista can see any files I put on it) then I partitioned the remaining 3GB as ext4 with parted.

Install via VirtualBox

No need to burn an ISO and reboot, use VirtualBox and do it from your desktop. You can follow my Testdrive a LiveCD with VirtualBox post to getting VirtualBox setup. I personally used Arch Linux for this install because it’s easy to configure.

Note: At the time VirtualBox does not have 64bit capabilities. If you want to install a 64bit Linux on your flash drive best to boot a LiveCD and follow these instruction from there.

Make sure your user is part of the VirtualBox group to enable usb recognition:

sudo gpasswd -a <username> vboxusers

Boot the LiveCD/USB iso/img in VirtualBox then in Devices > USB devices select your flash drive. Now the installer will recognize your flash drive. Proceed to install the distro on the flash drive. If you partitioned beforehand you can skip partitioning and go to setting Filesystem Mountpoints. When you reach GRUB setup be sure to install GRUB on the flash drive itself, for me it was /dev/sdb. Be sure NOT to install GRUB to a partition, it should be at the beginning of the drive.

Fix Grub

Because your BIOS is likely setup to recognize your hard drive before your USB drive you get drive denominations like /dev/sda for your hard disk and /dev/sdb for your flash drive on regular bootup. If booting from a flash drive, many BIOS’s have you enter a key (mine is F10) to get to a Boot Menu. So when you select your flash drive in your BIOS Boot Menu your flash drive now becomes /dev/sda, hard drive /dev/sdb. In grub terminology this is hd0 and hd1. Most BIOS’s are like this (though there a few exceptions). To know for sure you won’t be able to detect this until you try and boot your flash drive (more below).

Close VirtualBox and open your GRUB menu list and change to the first recognized drive:

sudo mount /dev/sdb2 /mnt/usb
sudo vim /mnt/usb/boot/grub/menu.lst

or however you edit your system files. Then change:

# (0) Arch Linux
title  Arch Linux
root   (hd1,1)
kernel /boot/vmlinuz26 root=/dev/disk/by-uuid/34393cdf-9f39-431e-88c8-ea89a2518c83 ro
initrd /boot/kernel26.img

to:

# (0) Arch Linux
title  Arch Linux
root   (hd0,1)
kernel /boot/vmlinuz26 root=/dev/disk/by-uuid/34393cdf-9f39-431e-88c8-ea89a2518c83 ro
initrd /boot/kernel26.img

The (hd0,1) value denotes the partition number, again starting with 0. So this denotation tells GRUB the root filesystem is on the first drive, second partition.

Arch-specific Details (Mostly)

If you already did the configuration for your hard disk, you should be able to copy most the configuation files over to the flash drive (rc.conf, mirrorlist, modprobe.conf, local.conf…) and then install xorg, xfce4… by chrooting in. This is my chroot script:

#!/bin/bash
# chrootmount - change root to current directory

cp /etc/resolv.conf etc/resolv.conf
mount -t proc none proc
mount -o bind /dev dev
mount -t sysfs none sys
chroot . /bin/bash

source /etc/profile
grep -v rootfs /proc/mounts > /etc/mtab
source ~/.bashrc

This will allow you to just cd to the mounted directory and enter command to chroot to the new environment. From there you can install a desktop environment (I choose XFCE because I wanted a lightweight environment and limited disk space):

pacman -Syu
pacman -S xorg xfce4 gdm <few-fonts> nvidia

And a couple other things following the Beginner’s Guide.

The kernel initramfs image will need to be rebuilt too to have usb driver support. In the chrooted environment edit /etc/mkinitcpio.conf and add usb to HOOKS:

HOOKS="base udev autodetect pata scsi sata filesystems usb"

Then find the the kernel version name and version:

uname -r

and build a new initramfs image:

mkinitcpio -g /boot/kernel26.img -k <your-kernel-name-version>

The -k option needs to be specified to use the chrooted kernel and not runtime kernel that is being used by chroot.

When done, exit chroot:

exit && umount proc sys dev

Reboot and Test

Now reboot and get to the BIOS Boot Menu. As I said, all BIOS’s are different so keep an eye for a key to get to it. Once in the Boot Menu select your USB drive.

Try and boot the flash drive. If you get a GRUB 17 error or boot into hard drive OS, you’ll have to edit your menu.lst. You can find the devices Grub sees by starting the flash drive again and in the Grub menu press e to edit. On the root line press e again and delete to:

root (hd

now press tab and it will show you the availble drive and partitions. Enter the correct one, hit escape and then b to boot. That’s it, you should now have your own customizable Linux USB drive.

Troubleshooting

If you get errors loading the kernel, it may be because USB device detection may need a delay before loading root. Try to add this to the end of your kernel line in your menu.lst:

rootdelay=8

Conclusion

I was a bit surprised. I didn’t think a USB drive would be much different that a CD/DVD but actually it was alot faster. And I just discovered that I’m using a USB 1.1 flash drive. :) Not quite as quick as my hard drive but definitely not bad. This is also the first time I ran without an xorg.conf and my desktop runs great. Definitely worth a try if you ever need a rescue os to fix problems with.

Installing OpenBSD 4.4

Following the ‘not enough time on my hands’ FreeBSD dive with a Power Mac, I decided to continue on BSD tourbus to OpenBSD. A good friend recommended it to me in the middle of my FreeBSD install and I’d of like to heard about it alot sooner. OpenBSD turned out to be a really great os, particularly for a router/firewall. I now have OpenBSD running successfully and it’s running good. The install does take a bit of work but the results are b-e-a-utiful.

OpenBSD is a shootoff of NetBSD (Berkly Software Distribution) that focuses on security – the code that is in OpenBSD is carefully audited. This guide will walk you through a basic install of OpenBSD (a few parts are macppc specific) and adds parts if you’d like to make OpenBSD a router.

Dual Boot MacOS?

The OpenBSD installer only has support for fdisk and not mac-fdisk so if you plan to dual-boot MacOS you should partition either with mac-fdisk (see the FreeBSD install link above), or with a Mac OS util like Disk Setup. Create one partition for Mac OS and another for OpenBSD. However, this install focuses on a whole disk install of OpenBSD.

Beginning with the installer

You might want to use OpenBSD’s Installation Guide as a companion guide along with this.

Put in the CD on a mac – you’ll have to start the cd from the Open Firmware prompt. Hold (Apple + option + O + F). at boot and at the OF prompt, type:

boot cd:,ofwboot 4.4/macppc/bsd.rd

The release cd will allow you to install, upgrade or use the shell:

Proceed? [y]

You will now initialize the disk(s) that OpenBSD will use. To enable all available security features you should configure the disk(s) to allow the creation of separate filesystems for /, /tmp, /var, /usr, and /home.

Available disks are: wd0 sd0.
wd0

wd0 must be partitioned using an HFS or an MBR partition table. If you plan to dual-boot MacOS use HFS, otherwise choose MBR.
MBR

Which one is the root disk (or ‘done’) [done] [wd0]
wd0
Do you want to use *all* of wd0 for OpenBSD? [no] y
Creating Master Boot Record (MBR)…done
/dev/rwd01

If you say no, you will be taken to fdisk where all you’ll be able to do is to type and label partitions. OpenBSD uses a two layer disk partitioning system: ‘fdisk’ and ‘disklabel’. If you are using the whole disk fdisk will be automatically configured.

‘disklabel’ is a tool to create partitions.

Initial label editor (enter ‘?’ for help at any prompt)

> p g

# size offset fstype [fsize bsize cpg]
c: 9.6G 0.0G unused 0 0
i: 0.0G 0.0G MSDOS

> a a # add ‘a’ partition
offset: [3024] <enter>
size: [20062224] 3.5g
FS type: [4.2BSD] <enter>
mount point: [none] /

> a b
offset: [17829504] <enter>
size: [2235744] 1.1g
FS type: [swap] <enter>

> a d
<enter>
3.0g
<enter>
/usr

> a e
<enter>
1.0g
<enter>
/tmp

> a f
<enter>
<enter> # to end of drive
<enter>
/var

> p g
OpenBSD area: 0.0G-9.6G; size 9.6G; free: 0.0G
# size offset fstype [fsize bsize cpg]
a: 3.5G 0.0G 4.2BSD 2048 16384 1 # /
b: 1.1G 3.5G swap
c: 9.6G 0.0G unused 0 0
d: 3.0G 4.6G 4.2BSD 2048 16384 1 # /usr
e: 1.0G 7.6G 4.2BSD 2048 16384 1 # /tmp
f: 1.0G 8.6G 4.2BSD 2048 16384 1 # /var
i: 0.0G 0.0G MSDOS

> w # write
> q # quit

Mount point for wd0d (size=3072Mbytes)? (or ‘none’ or ‘done’) [/usr] <enter>
Mount point for wd0e (size=4097144k)? (or ‘none’ or ‘done’) [/tmp] <enter>
Mount point for wd0f (size=6291432k)? (or ‘none’ or ‘done’) [/var] <enter>
Mount point for wd0d (size=3072Mbytes)? (or ‘none’ or ‘done’) [/usr] done

No label changes.
Available disks are: sd0.
Which one do you wish to initialize? (or ‘done’) [done] done

The next step *DESTROYS* all existing data on the partition!
Are you really sure that you’re ready to proceed? [no] y

System hostname (short form, e.g. ‘foo’): dirk-pmac

Configure the network? [yes] <enter>

You’ll be shown available Network Interface Cards (NIC)s. If you’re building a router, you’ll have two. The first one will get it’s address via the ISP DHCP server. On the second one assign a LAN address like 192.168.111.7.

If you choose dhcp, OpenBSD install will try to get a lease from the DHCP server. I didn’t bother connecting the cables because InstallerCD’s are notably insecure and the installer doesn’t need it anyway.

Enter your Domain Name Servers (separated by a space) and password for root account. Choose your install sets, the defaults will give you a basic system.

Location of sets? (cd disk ftp http or ‘done’) [done] <enter>

sshd yes
ntpd yes
ntp server us.pool.ntp.org

Change the default console to com0? [no] <enter>

Enter timezone, and then your done with the basics.

Afterboot Tasks

Reboot and start the Open Firmware prompt and boot OpenBSD by:

boot hd:,owfboot /bsd

There are a few tasks that need to be done to finish the install following the afterboot manpage.

errata check.

If a reliability or security issue effects you, you will have to patch your system – a detailed proposition which I will detail in another post.

check ‘date’, ‘hostname’, networking will be done in a bit.

Add new user

adduser
Enter your default shell: csh ksh nologin sh [ksh]: <enter>
# Default login class defines allocation of system-resources, and
# environment setup.
Default login class: authpf daemon default staff [default]: <enter>
Enter your default HOME partition: [/home]: <enter>
Copy dotfiles from: /etc/skel no [/etc/skel]: <enter>
Send message from file: /etc/adduser.message no [no]: <enter>
Prompt for passwords by default (y/n) [y]: <enter>
Default encryption method for passwords: auto blowfish des md5 old [auto]: <enter>
Don’t worry about mistakes. There will be a chance later to correct any input.
Enter username []: Dirkgen2ly
Enter full name []: Dirk Gently
Enter bash csh ksh nologin sh [ksh]: <enter>
Uid [1000]: 1222
Login group Dirkgen2ly [Dirkgen2ly]: <enter>
# Add to wheel group to allow ‘su’ to root.
Login group is “Dirkgen2ly”. Invite into other groups: guest no
[no]: wheel
Login class authpf daemon default staff [default]: <enter>
Enter password []:
Enter password again []:

Configuring sendmail

Set-mailserver aliases in /etc/mail/aliases:

vi /etc/mail/aliases

# Well-known aliases — these should be filled in!
root: root
manager: root
dumper: root

Run ‘newaliases’ to update sendmail aliases.

A sendmail-configuration file will need to be built, from papamike:

Sendmail configuration files are built with a macro-processor. A macro-processor is basically a program that scans text looking for defined symbols, which it replaces by other text — or other symbols. The one used with Sendmail is called m4… So m4 inputs a macro configuration file, with extension .mc, and outputs a sendmail configuration file to standard output. Typically we redirect this output to a file, with extension .cf

Luckily there are some examples. If not planning to use sendmail externally (i.e. to the internet) use openbsd-localhost.cf.

cp /usr/share/sendmail/cf/openbsd-localhost.mc /usr/share/sendmail/cf/localhost.mc
m4 /usr/share/sendmail/m4/cf.m4 /usr/share/sendmail/cf/localhost.mc > /etc/mail/localhost.cf

Then test it:

sendmail -v -t -C /etc/mail/localhost.cf
^C

A valid config will give no output. Now tell the the sendmail daemon to load the configuration file at boot in /etc/rc.conf.local:

sendmail_flags=”-L sm-mta -C/etc/mail/localhost.cf -bd -q30m”

The -C/ is necessary, it’s not a typo.

Daily, weekly, monthly scripts

Run the daily and weekly scripts to make sure they run alright.

sh /etc/daily
sh /etc/weekly

Warning: When running the weekly script if you get:

Rebuilding locate database:
Not installing locate database; zero size

Try “vi /var/db/locate.database“, put a space in, save it “:x”, and run the weekly script again.

Tighten up security

Covered below.

File Systems

OpenBSD doesn’t have a journaled file-system, meaning that you could lose critical disk data in event of a crash. Rather OpenBSD has incorporated soft updates a userland program that performs a likewise task that also improves disk performance by utilizing a cache. Adding the softdep option to each ffs partition in /etc/fstab will enable soft updates at next boot.

# perl -pi -e ‘s/ffs rw/ffs rw,softdep/’ /etc/fstab
# cat /etc/fstab
/dev/wd0a / ffs rw,softdep 1 1
/dev/wd0e /tmp ffs rw,softdep,nodev,noexec,nosuid 1 2
/dev/wd0d /usr ffs rw,softdep,nodev 1 2
/dev/wd0f /var ffs rw,softdep,nodev,nosuid 1 2
/dev/cd0c /cdrom cd9660 ro,noauto 0 0

CD/DVD define mount point

mkdir /mnt/cdrom
echo “/dev/cd0c /cdrom cd9660 ro,noauto 0 0” >> /etc/fstab

Networking

Following the Network FAQ. The installer will have created /etc/hostname.<NIC> for each device you have. Make sure they are correct:

cat /etc/hostname.*

For a router, ip-forwarding will need to be enabled in /etc/sysctl.conf:

net.inet.ip.forwarding=1

Enter your DNS servers in /etc/dhclient.conf:

request subnet-mask, broadcast-address, routers, domain-name,
domain-name-servers, host-name;
supersede domain-name-servers <domain-name-server1>, <domain-name-server2>;

If you are planning to be testing a firewall and disconnecting your internet-connection from time to time, its good to up dhclients timeout. ‘dhclient’ rechecks it’s connection to the DHCP server on an exponential scale if it is unable to locate it it will take down the WAN NIC. Add to dhclient.conf and get larger timeouts as time goes by:

timeout 216000;
retry 216000;
link-timeout 216000;

To be able to connect the LAN PC to the network this guide uses dnsmasq (a good solution for small networks) but first it is a good idea to setup firewall and close unused ports.

Enabling a Firewall

Building a firewall is a necessary evil, even on the most secure of systems networking will be the greatest security-hole. Here are a few tips for working with PF:

Enable PF at Boot:

echo “pf=YES # PF Firewall” >> /etc/rc.conf.local

‘ftp-proxy’ will need to be enabled to ftp past a firewall, first enable it at boot:

echo “ftpproxy_flags=”” # ftp-proxy daemon” >> /etc/rc.conf.local

Enable ftp-proxy in the NAT section of your pf.conf:

nat-anchor “ftp-proxy/*”
rdr-anchor “ftp-proxy/*”
rdr on $LAN_NIC proto tcp from any to any port 21 -> 127.0.0.1 port $FTPPORT

And in the filter section, anchor ftp and allow pass out:

anchor “ftp-proxy/*”
pass out quick on $WAN_NIC proto tcp from $WAN_NIC to any port {20,21} flags S/AUPRFS modulate state

Another good idea is when writing block-rules to log them to be able to test the firewall and to see if there are any attempts to attack the firewall:

block in log on $WAN_NIC all

To check your PF configuration for errors, run:

pfctl -nf /etc/pf.conf

A couple other commands:

pfctl -e # Enable PF – Enable but will not load ruleset.
pfctl -d # Disable PF
pfctl -f /etc/pf.conf # load the rules
pfctl -sn # Show the current NAT rules
pfctl -sr # Show the current filter rules
pfctl -ss # Show the current state table
pfctl -si # Show filter stats and counters
pfctl -sa # Show EVERYTHING it can show
pfctl -F all # Flush all rules, nat, states, options, tables

To test the firewall in real time, run ‘pflogd’ then:

tcpdump -n -e -ttt -i pflog0

To have pflog load at boot:

echo “pflogd_flags=”” # pflog device” >> /etc/rc.conf.local

You may have to reboot to have pflog0 show up in ifconfig. Now that the firewall is up you can start (or restart) the network:

sh /etc/netstart

Adding a Package

OpenBSD has two ways to add software: package and ports. Packages are pre-built binaries that can be downloaded and quickly installed onto your system and are the recommended way to add software to your system. OpenBSD also has a port system that contains information necessary to build packages and their dependencies from source.

OpenBSD has many prebuilt packages for a number of different architectures. Find a nearby mirror and add it to ~/.profile:

export PKG_PATH=ftp://<your.ftp.mirror>/pub/OpenBSD/`uname -r`/packages/`machine -a`/

Those are back ticks BTW. `uname -r` adds your release version of OpenBSD you are using and `machine -a` will be your architecture. To source (reload) your .profile so the variable is known to the korn shell:

. ~/.profile

Then add a package:

pkg_add -v <packagename>

A couple other package commands:

pkg_info # Show installed packages
pkg_delete -n # Delete applications and their dependencies.

Configure the LAN

Now that you can add a package you can add dnsmasq and get your router going. It is possible to define a static-route and not to have to use a DHCP server to define an address and route, but using a DHCP server makes the job tons easier. dnsmasq is a great lightweight application that will provide a route to and from the LAN machine. dnsmasq is also provides a DNS cacher to make resolving of domain names very very fast.

If planning on creating a LAN of > 50 machines you should use the pre-installed dhcpd.

pkg_add -v dnsmasq

Edit /etc/dnsmasq.conf:

# Only listen to WAN box LAN NIC and use local loopback for DNS caching.
interface=<your-router-LAN-NIC>

# dnsmasq will open TCP port 53 and UDP port 67 to world
# tohelp with dynamic interfaces. dnsmasq will discard
#requests to them, but I like better not to have these
#ports open and let the kernel handle it.
bind-interfaces

# Dynamic range of IPs to make available
dhcp-range=192.168.111.50,192.168.111.100,12h

# If you’d like to have a static ip, bind the LAN computers
# NIC MAC address
dhcp-host=aa:bb:cc:dd:ee:ff,192.168.111.22

dhclient.conf will need to be edited again to know that dnsmasq is handing dns requests. Redirect dhclient to localhost and dnsmasq will take it from there.

request subnet-mask, broadcast-address, routers, domain-name,
domain-name-servers, host-name;
supersede domain-name-servers 127.0.0.1, <dns1>, <dns2>;

dhclient by default appends to /etc/resolv.conf details to use the pre-installed BIND name server assuming people are going to use it. BIND isn’t enabled by default and since dnsmasq is handling this, comment out “lookup file bind” in /etc/resolv.conf.tail and restart the network:

sh /etc/netstart

cat /etc/resolv.conf
search eau.wi.charter.com
nameserver 127.0.0.1
nameserver 208.67.222.222
nameserver 208.67.222.222
#lookup file bind

Now you can start dnsmasq:

dnsmasq

To load at boot put in rc.local:

# Start Dnsmasq
if [ -x /usr/local/sbin/dnsmasq ]; then
echo -n ‘ dnsmasq’; /usr/local/sbin/dnsmasq
fi

To have your LAN computer connect to your router set it to dhcp and connect.

Test DNS caching:

dig https://linuxtidbits.wordpress.com | grep “Query time”

Do it again and you’ll notice a faster lookup.

NTP

Many NTP configurations default to pool.ntp.org which is great for a whole list to choose from from the entire world, but it’s better to use something local ;). Add to /etc/ntpd.conf:

server us.pool.ntp.org

Because ntpd slowly adjusts the clock if it’s off you can add to crontab entry to get it fixed daily:

crontab -e

# Update date and time daily
32 1 * * * ntpd -s

Securing the Network

A good firewall will close ports but some applications may try to open them again. Best to close any ports you don’t need. Run netstat and get a good idea of what’s open:

netstat -ant | egrep ‘udp | LISTEN’

Shows open ports:

TCP – 13 37 22 113
UDP – 514

To find out what these ports do:

grep <port> /etc/services

daytime, time, ssh, auth, and syslog (udp). You can find more infomation about the port (like the program that opened it) with:

lsof -i | grep 22

Most people don’t use daytime, time, auth anymore and can be safely disabled in /etc/rc.conf.local:

inetd=NO

The Syslog port can not be turned off in inetd. It is invoked in rc.conf with no “-u” flag meaning that it is listening on UDP port 514 but that incoming packets are ignored, it is only used to send. You’ll will also see a couple ports open for tcp6 (ipv6) like ::1.587 or ::1.25 these are loopbacks (local) for ipv6 and will be secure from the outside world.

Using nmap is an ever more reliable way to test for open ports. Now that the firewall is up, you might want to add nmap and test it.

nmap -p1-65535 -T4 -sS <WAN-IP>

SSH Daemon

SSH opens port 22 to world by default, If you don’t plan on accessing from outside you can bind it to your LAN computer in /etc/ssh/sshd_config:

ListenAddress <Router LAN_NIC Address>

Also root login is a bad idea, since the regular user is able to su this is a good idea to define:

PermitRootLogin no

Add a key so trusted computers can connect:

ssh-keygen -t dsa

Use a password that is not your login password. Name the output something useful like powermac-dsa-key. These keys by default are generated into the local ~/.ssh/ directory but belong in the accessee’s ~/.ssh directory. Move them to the LAN computer ~/.ssh to be a trusted computer.

Note: SSH attempts to resolve an address even if it’s local through /etc/resolv.conf. When Domain Name Server isn’t setup yet or a DNS hangs, ssh will hang at “debug1: SSH2_MSG_SERVICE_ACCEPT received” you will have to wait about a minute until ssh decides no DNS can be reached.

Wrap-up

That should get you a good start.

PCI, PCI-X, PCI Express – Oh boy!

Lately I bought an old pc to use as a server and needed a network card for it. I didn’t think it be such a hassle but because of multiple PCI specs finding a card wasn’t easy. Theres been alot of confusion about pci cards and what card to get for your computer – PCI cards come in alot of different types and versions. I’ve done a good amount of research on this (if there are any discrepancies, please let me know) and hopefully this post will help clear things up.

PCI

Standard PCI cards (sometimes called pci 1.0) have a 32 bit width slot, and operate at 33 MHz. Originally they started as 5 volt cards but 3.3 volt cards began to be made that use a different slot.

PCI 2.1 came a few years later that added the Universal PCI card spec that allowed cards to be used in both 3.3 and 5v slots, and upped the bus to 66 MHz. Also they created a pci 64 bit width slot for high end cards (gigabit networking,…). This meant that there could be one of 4 different slots in your computer: 5v 32bit, 3.3v 32bit, 3.3v 64bit, 5v 64bit (see graphic below). This meant you either had to buy an exact card for the slot or a universal card (which most manufactures began to build).

The PCI bus 2.3 spec came along and nix’d 5v adapters (cards). PCI 2.3 was adaptable though and supported 3.3v cards and universal pci cards.

PCI-X

PCI-X or PCI eXtended was built mainly for high end use. It has a bus speed of 66 or 133 MHz and only used the 64 bit 3.3v slot. It is fully backward compatible though with the existing PCI architecture: 33/66 MHz PCI adapters (cards) can be used in PCI-X slots and PCI-X adapters can be used in PCI slots. PCI-X 2.0 came along and really upped the bus speed to either 266 MHz or 533 MHz, but was still fully backwards compatible.

Which Card to Get?

Well really you can get any universal card and have it work. Carnildo helped me see things the easy way:

The rule of thumb for PCI and PCI-X cards is that if it fits in the slot, it’ll work. The bus and cards will negotiate the fastest, widest connection that all of them can use, so a 133MHz 64-bit card in a standard PCI slot will transfer data as if it were a 33MHz 32-bit PCI card.

Also keep in mind that, “The slowest board dictates the maximum speed on a particular bus!”

PCI Express?

PCI Express uses an entirely different architecture, different slot sizes, and is incompatible with with PCI or PCI-X. It’s expected to coexist with PCI-X and not replace it.