Showing posts with label debian testing. Show all posts
Showing posts with label debian testing. Show all posts

02 July 2013

471. Debian Jessie -- gnome-shell bug

Update 3/7/2013:
there are now *gnome-bluetooth packages (3.8.1-2) in the jessie repos now. While I haven't looked closer at them, I presume that they fix this issue.

(on a different note: dist-upgrade currently removes gnome...)

Original post:
I've used debian testing since early 2011, and I've only had a few minor issues during that time.

However, sometimes things happen that reminds you that the Testing release is not meant for mission critical work (and makes me happy that I only use Jessie on my laptop, which I mainly use at home).

So...

Last night I did upgrade and dist-upgrade, which installed the following packages according to /var/log/apt/history:
Start-Date: 2013-07-01  22:03:17
Commandline: apt-get dist-upgrade
Install: p11-kit:amd64 (0.18.3-2, automatic), libgnome-bluetooth11:amd64 (3.8.1-1, automatic), libgcr-base-3-1:amd64 (3.8.2-3, automatic), libtasn1-6:amd64 (3.3-1, automatic), libgcr-ui-3-1:amd64 (3.8.2-3, automatic)
Upgrade: libnm-gtk0:amd64 (0.9.8.2-1, 0.9.8.2-1+b1), libgcr-3-1:amd64 (3.4.1-3, 3.8.2-3), gir1.2-gcr-3:amd64 (3.4.1-3, 3.8.2-3), network-manager-gnome:amd64 (0.9.8.2-1, 0.9.8.2-1+b1), gnome-keyring:amd64 (3.4.1-5, 3.8.2-2), gcr:amd64 (3.4.1-3, 3.8.2-3), gnome-bluetooth:amd64 (3.4.2-1, 3.8.1-1), gir1.2-gnomebluetooth-1.0:amd64 (3.4.2-1, 3.8.1-1), gir1.2-gck-1:amd64 (3.4.1-3, 3.8.2-3)
End-Date: 2013-07-01  22:03:29

Now what happens when I log in to gnome via gdm3 I get an empty desktop with no menus, no hot-spots or anything else indicating that things worked out. Alt+F2 doesn't work either, and conky doesn't start.

The only thing that does work is
* my keyboard shortcuts (I've mapped ctrl+shift+Down arrow to chromium)
* guake (which starts with gnome)

ps aux|grep gnome-shell
returns nothing, which might be a clue.

Looking at the debian forums the closest post seems to be (although erroneously labelled -- gdm3 DOES start): http://forums.debian.net/viewtopic.php?f=6&t=105393&p=504077&hilit=gnome+shell#p504077

That in turn led to this bug report: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=712861

My gnome-shell version is 3.4.2-8,

I don't understand how gnome-bluetooth causes this, especially given that I've disabled bluetooth in rcconf, but whatever it takes...

I tried applying the patch but it failed:
mkdir ~/tmp
cd ~/tmp
wget "http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=66;filename=GnomeBluetooth.patch;att=1;bug=712861" -O blue.patch
sed -i 's_js/ui/status/bluetooth.js_/usr/share/gnome-shell/js/ui/status/bluetooth.js_g' blue.patch
sudo patch -p0 < blue.patch

Instead, I ended up making the changes to /usr/share/gnome-shell/js/ui/status/bluetooth.js by hand (remember that you can always use the ttys using ctrl+Fx):
  6 const Gio = imports.gi.Gio;
  7 const GnomeBluetoothApplet = imports.gi.GnomeBluetoothApplet;
  8 const GnomeBluetooth = imports.gi.GnomeBluetooth;
  9 const Gtk = imports.gi.Gtk;

and then delete the Applet part in GnomeBluetoothApplet so that it reads
 38         this._killswitch.connect('toggled', Lang.bind(this, function() {
 39             let current_state = this._applet.killswitch_state;
 40             if (current_state != GnomeBluetooth.KillswitchState.HARD_BLOCKED &&
 41                 current_state != GnomeBluetooth.KillswitchState.NO_ADAPTER) {
 42                 this._applet.killswitch_state = this._killswitch.state ?
 43                     GnomeBluetooth.KillswitchState.UNBLOCKED:
 44                     GnomeBluetooth.KillswitchState.SOFT_BLOCKED;
 45             } else
 46                 this._killswitch.setToggleState(false);

Then do it again:
 96     _updateKillswitch: function() {
 97         let current_state = this._applet.killswitch_state;
 98         let on = current_state == GnomeBluetooth.KillswitchState.UNBLOCKED;
 99         let has_adapter = current_state != GnomeBluetooth.KillswitchState.NO_ADAPTER;
100         let can_toggle = current_state != GnomeBluetooth.KillswitchState.NO_ADAPTER &&
101                          current_state != GnomeBluetooth.KillswitchState.HARD_BLOCKED;
102 



At this point I rebooted and everything was back to normal (you can try simply doing 'sudo service gdm3 restart' instead of rebooting).
Anyway, done.

25 April 2013

395. Compiling the ck-kernel 3.8.x on debian (and patching i915)


What is the ck kernel? In short it is a patch for the mainline kernel which is supposed to render it more responsive for desktop use. See here: https://wiki.archlinux.org/index.php/Linux-ck

I'm not going to try to 'benchmark' this against the stock kernel, since I'm not clear what a good metric would be -- how do you measure 'responsiveness'? Come to think of it -- what's a good, objective definition?

Download the kernel version that you need from http://kernel.org and the ck path from Con's web page at http://users.on.net/~ckolivas/kernel/

I also ended up applying this patch to sort out recent i915 issues:
http://verahill.blogspot.com.au/2013/03/368-slow-mouse-and-keyboard-triggered.html
https://bbs.archlinux.org/viewtopic.php?pid=1248190
http://forums.gentoo.org/viewtopic-p-7278760.html
https://bbs.archlinux.org/viewtopic.php?pid=1254285#p1254285

I did it by creating a file called i915.patch and placing it in the kernel source root:
--- drivers/gpu/drm/drm_crtc_helper.c 2013-04-26 10:24:07.987942008 +1000 +++ ../../linux-3.8.8/drivers/gpu/drm/drm_crtc_helper.c 2013-04-17 15:11:28.000000000 +1000 @@ -1067,7 +1067,7 @@ void drm_helper_hpd_irq_event(struct drm enum drm_connector_status old_status; bool changed = false; - if (!dev->mode_config.poll_enabled || !drm_kms_helper_poll) + if (!dev->mode_config.poll_enabled) return; mutex_lock(&dev->mode_config.mutex);
Note that as far as I know it doesn't solve the issue, but it does allow you to disable polling by editing /etc/default/grub and adding
GRUB_CMDLINE_LINUX_DEFAULT="quiet drm_kms_helper.poll=N"
followed by running
sudo update-grub

So far it's working fine (after testing for ca one hour).

Anyway, here's kernel 3.8.5 (should work on all 3.8.x):
sudo apt-get install kernel-package fakeroot build-essential ncurses-dev
mkdir ~/tmp
cd ~/tmp
wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.8.5.tar.xz
tar xvf linux-3.8.5.tar.xz
cd linux-3.8.5/
wget http://ck.kolivas.org/patches/3.0/3.8/3.8-ck1/patch-3.8-ck1.bz2
bunzip2 patch-3.8-ck1.bz2
patch -p1 < patch-3.8-ck1
patching file arch/powerpc/platforms/cell/spufs/sched.c patching file Documentation/scheduler/sched-BFS.txt patching file Documentation/sysctl/kernel.txt patching file fs/proc/base.c patching file include/linux/init_task.h patching file include/linux/ioprio.h patching file include/linux/sched.h patching file init/Kconfig patching file init/main.c patching file kernel/delayacct.c patching file kernel/exit.c patching file kernel/posix-cpu-timers.c patching file kernel/sysctl.c patching file lib/Kconfig.debug patching file include/linux/jiffies.h patching file drivers/cpufreq/cpufreq.c patching file drivers/cpufreq/cpufreq_ondemand.c patching file drivers/cpufreq/cpufreq_conservative.c patching file kernel/sched/bfs.c patching file kernel/sched/Makefile patching file include/uapi/linux/sched.h patching file include/linux/swap.h patching file mm/memory.c patching file mm/swapfile.c patching file mm/vmscan.c patching file arch/x86/Kconfig patching file kernel/Kconfig.hz patching file kernel/Kconfig.preempt patching file Makefile
patch -p0 -R < i915.patch
patching file drivers/gpu/drm/drm_crtc_helper.c
make-kpgk clean cat /boot/config-`uname -r`>.config make oldconfig time fakeroot make-kpkg -j4 --initrd kernel_image kernel_headers sudo dpkg -i ../linux*3.8.5-ck*.deb

where 4 is the number of cores on your machine (note: it only has to do with compiling -- you can use the compiled binaries on any number of cores).

That's it. Now all you have to do is reboot and you'll be using your new kernel. Simple, eh?

18 April 2013

391. encfs on debian testing

Truecrypt is nice and all, but the way it's managed is a bit un-linux like. It IS open source though, but because it isn't completely clear who the maintainers are, and while the source code is probably safe to use, most people use pre-built binaries -- just because it's open source doesn't mean that the binaries are built from that source. I'm happy with truecrypt, but if we're going to be paranoid, we might as well go all the way. Also, it would be nice with an encryption tool which can be installed from the repos, and as a binary built by a trusted party (i.e. the debian maintainers).

Note that truecrypt wins hands down on features -- as always, pick the tool that does the job.

encfs is a well-established encryption tool for linux systems and is available in most distro repos -- it does encryption of files, and while it lacks many of the options of truecrypt, it is useful for encrypting directory structures.

Installation
sudo apt-get install encfs

Yes, that's it. If you're having issues, make sure that fuse is installed, and that you belong to the group fuse (too add, do sudo usermod -a -G fuse $USER . To check do cat /etc/group|grep fuse)

Set-up
Set-up is equally easy.
 encfs ~/.secret ~/encrypted
The directory "/home/me/encrypted" does not exist. Should it be created? (y,n) Y Creating new encrypted volume. Please choose from one of the following options: enter "x" for expert configuration mode, enter "p" for pre-configured paranoia mode, anything else, or an empty line will select standard mode. ?> Standard configuration selected. Configuration finished. The filesystem to be created has the following properties: Filesystem cipher: "ssl/aes", version 3:0:2 Filename encoding: "nameio/block", version 3:0:1 Key Size: 192 bits Block Size: 1024 bytes Each file contains 8 byte header with unique IV data. Filenames encoded using IV chaining mode. File holes passed through to ciphertext. Now you will need to enter a password for your filesystem. You will need to remember this password, as there is absolutely no recovery mechanism. However, the password can be changed later using encfsctl. New Encfs Password: Verify Encfs Password:

At the end of this your encryption folder will be mounted.

How it works:
A (to me) nice way of illustrating how it works is the following:
ls ~/encrypted/ -lah
total 24K drwxr-xr-x 2 me me 4.0K Apr 18 14:29 . drwxr-xr-x 212 me me 20K Apr 18 14:29 ..
ls ~/.secret/ -lah
total 32K drwxr-xr-x 2 me me 4.0K Apr 18 14:29 . drwxr-xr-x 212 me me 20K Apr 18 14:29 .. -rw-r--r-- 1 me me 1.1K Apr 18 14:29 .encfs6.xml -rw-r--r-- 1 me me 411 Nov 14 2011 .passwd.gpg
touch ~/encrypted/test ls ~/encrypted/ -lah
total 24K drwxr-xr-x 2 me me 4.0K Apr 18 14:32 . drwxr-xr-x 212 me me 20K Apr 18 14:29 .. -rw-r--r-- 1 me me 0 Apr 18 14:32 test
ls ~/.secret/ -lah
total 32K drwxr-xr-x 2 me me 4.0K Apr 18 14:32 . drwxr-xr-x 212 me me 20K Apr 18 14:29 .. -rw-r--r-- 1 me me 1.1K Apr 18 14:29 .encfs6.xml -rw-r--r-- 1 me me 0 Apr 18 14:32 IfSGAzSkFsuN2A,SpPmnKSYu -rw-r--r-- 1 me me 411 Nov 14 2011 .passwd.gpg
df -h
encfs 734G 525G 172G 76% /home/me/encrypted

In other words, anything that is put into ~/encrypted ends up as an encrypted file in ~/.secret. It's a little bit different from truecrypt, which uses a filesystem container -- this is more similar to encrypting each file with gpg/pgp, just with a good way of managing it from a filesystem pov.

Also, this means that you don't have to dedicate a certain amount of space to the encrypted file system as you would have to with truecrypt.

Anyway, unmount ~/encrypted and no-one will have access to the decrypted forms of the files.
sudo umount ~/encrypted

To mount it again, just do
encfs ~/.secret ~/encrypted

You can change the name of the directory you want to mount it in.

It is also possible to mount automatically on boot, but this would defeat most purposes of encrypting in the first place.

15 April 2013

387. Compiling Wine 1.5.28 in a i386 chroot on Debian Wheezy amd64

UPDATE 16 May 2013: See here for Wine 1.5.30: http://verahill.blogspot.com.au/2013/05/416-wine-1530-in-chroot.html

UPDATE: for those who don't want to compile and are willing to take a minor risk* there are pre-built debs for mepis here: http://main.mepis-deb.org/mepiscr/testrepo/pool/test/w/wine/
They are provided by Steve Pusser, who is an active member at the debian forums.

* I presume, but don't actually know, that they are compatible with debian due to 1) Steve being active on the debian forums and 2) Mepis being based on debian stable (currently Squeeze)

Original post

New version of wine out now: 1.5.28.

Not much to get excited about from what I can tell, but wine now relies on GnuTLS instead of openssl.

Here's how to build it -- a lot of it is shamelessly recycled from earlier posts on this blog.

Getting started:
If you set up a chroot to build 1.5.27 before, you don't need to set up a new chroot to build 1.5.28. In that case, skip the set-up step below and instead re-enter your existing chroot like this:
sudo mount -o bind /proc wine32/proc
sudo cp /etc/resolv.conf wine32/etc/resolv.conf
sudo chroot wine32
su sandbox

Setting up the Chroot
mkdir $HOME/tmp/architectures/wine32
cd $HOME/tmp/architectures
sudo debootstrap --arch i386 wheezy $HOME/tmp/architectures/wine32 http://ftp.au.debian.org/debian/
sudo mount -o bind /proc wine32/proc
sudo cp /etc/resolv.conf wine32/etc/resolv.conf
sudo chroot wine32

You're now in the chroot:
apt-get update
apt-get install locales sudo vim
echo 'export LC_ALL="C"'>>/etc/bash.bashrc
echo 'export LANG="C"'>>/etc/bash.bashrc
echo '127.0.0.1 localhost beryllium' >> /etc/hosts
source /etc/bash.bashrc
adduser sandbox
usermod -g sudo sandbox
echo 'Defaults !tty_tickets' >> /etc/sudoers
su sandbox
cd ~/

Building Wine
While still in the chroot, continue (the i386 is ok; don't worry about it -- you don't actually need it):

sudo apt-get install libx11-dev:i386 libfreetype6-dev:i386 libxcursor-dev:i386 libxi-dev:i386 libxxf86vm-dev:i386 libxrandr-dev:i386 libxinerama-dev:i386 libxcomposite-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libdbus-1-dev:i386 libgnutls-dev:i386 libncurses-dev:i386 libsane-dev:i386 libv4l-dev:i386 libgphoto2-2-dev:i386 liblcms-dev:i386 libgstreamer-plugins-base0.10-dev:i386 libcapi20-dev:i386 libcups2-dev:i386 libfontconfig-dev:i386 libgsm1-dev:i386 libtiff-dev:i386 libpng-dev:i386 libjpeg-dev:i386 libmpg123-dev:i386 libopenal-dev:i386 libldap-dev:i386 libxrender-dev:i386 libxml2-dev:i386 libxslt-dev:i386 libhal-dev:i386 gettext:i386 prelink:i386 bzip2:i386 bison:i386 flex:i386 oss4-dev:i386 checkinstall:i386 ocl-icd-libopencl1:i386 opencl-headers:i386 libasound2-dev:i386 build-essential
mkdir ~/tmp
cd ~/tmp
wget http://prdownloads.sourceforge.net/wine/wine-1.5.28.tar.bz2
tar xvf wine-1.5.28.tar.bz2
cd wine-1.5.28/
./configure
time make -j2
sudo checkinstall --install=no
checkinstall 1.6.2, Copyright 2009 Felipe Eduardo Sanchez Diaz Duran This software is released under the GNU GPL. The package documentation directory ./doc-pak does not exist. Should I create a default set of package docs? [y]: Preparing package documentation...OK Please write a description for the package. End your description with an empty line or EOF. >> wine 1.5.28 >> ***************************************** **** Debian package creation selected *** ***************************************** This package will be built according to these values: 0 - Maintainer: [ root@beryllium ] 1 - Summary: [ wine 1.5.28 ] 2 - Name: [ wine ] 3 - Version: [ 1.5.28 ] 4 - Release: [ 1 ] 5 - License: [ GPL ] 6 - Group: [ checkinstall ] 7 - Architecture: [ i386 ] 8 - Source location: [ wine-1.5.28 ] 9 - Alternate source location: [ ] 10 - Requires: [ ] 11 - Provides: [ wine ] 12 - Conflicts: [ ] 13 - Replaces: [ ]

Compilation took ca 19 minutes. Checkinstall takes a little while (In particular this step: 'Copying files to the temporary directory...').

Installing Wine
Exit the chroot
sandbox@beryllium:~/tmp/wine-1.5.28$ exit
exit
root@beryllium:/# exit
exit
me@beryllium:~/tmp/architectures$ 

On your host system
Enable multiarch and install ia32-libs, since you've built a proper 32 bit binary:
sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install ia32-libs

Copy the .deb package and install it
sudo cp wine32/home/sandbox/tmp/wine-1.5.28/wine_1.5.28-1_i386.deb .
sudo chown $USER wine_1.5.28-1_i386.deb
sudo dpkg -i wine_1.5.28-1_i386.deb

Links to this post:
http://forum.winehq.org/viewtopic.php?f=8&t=18924

21 March 2013

366. nspluginwrapper on debian testing/wheezy

I only recently realised that nspluginwrapper wasn't in the debian wheezy/testing repos -- and it is absolutely necessary if you want to get BankID to work (here and here).

The easiest way to install nspluginwrapper in debian is to enable the stable repos. E.g. on my wheezy laptop I have the following /etc/apt/sources.list:
deb http://ftp.au.debian.org/debian/ wheezy main contrib non-free
deb http://ftp.au.debian.org/debian/ squeeze main contrib non-free
While normally you shouldn't mix different repos without some pretty strict apt-pinning policies, this works since apt automatically pulls in the newest version of each package -- and that'd be the package in wheezy which is the distro I'm using.

If that still makes you nervous, the second easiest option is simply to compile nspluginwrapper yourself. It's complicated a little bit by bug 682678.

sudo apt-get install libc6-dev-i386 g++-multilib checkinstall libglib2.0-dev build-essential libgtk2.0-dev libcurl4-gnutls-dev libxt-dev
mkdir ~/tmp
cd ~/tmp
wget http://nspluginwrapper.org/download/nspluginwrapper-1.4.4.tar.gz
tar xvf nspluginwrapper-1.4.4.tar.gz
cd nspluginwrapper-1.4.4/
./configure
make
sudo checkinstall


Error 1:
make fails with complaints about
/usr/include/features.h -> bits/predefs.h

It's fixed by installig libc6-dev-i386

Error 2:
make fails with a message about
skipping incompatible libsupc++.a

It's fixed by installing g++-multilib

365. Compile firefox 19 on Debian Testing/Wheezy

There are issues with the official firefox binary since it's a 32 bit build and Wheezy isn't handling multiarch properly yet (plenty of conflicts between 32 and 64 bit packages)

So here's how to build a proper 64 bit firefox 19 on debian testing:

sudo apt-get install bzip2 build-essential python zip unzip pkg-config libgtk2.0-dev gconf2 libdbus-1-dev libdbus-glib-1-dev yasm libasound2-dev libcurl4-gnutls-dev libxt-dev mesa-common-dev
mkdir ~/tmp/firefox -p
cd ~/tmp/firefox
wget ftp://ftp.mozilla.org/pub/firefox/releases/19.0.2/source/firefox-19.0.2.source.tar.bz2
rm mozilla-release -rf
tar xvf firefox-19.0.2.source.tar.bz2
mkdir buildfirefox19/
cd buildfirefox19/
../mozilla-release/./configure --prefix=$HOME/.firefox19
make
make install
echo "alias firefox19='$HOME/.firefox19/bin/firefox'" >> $HOME/.bashrc
source $HOME/.bashrc

Start by running firefox19 from the terminal.

02 February 2013

330. Installing Arch linux: installing from existing linux onto an external USB HDD

Update: You might want to look at this post instead: http://verahill.blogspot.com.au/2013/02/331-full-linux-install-on-usb-stick.html -- it covers setting up Arch with LXDE (very, very basic set-up -- more openbox than lxde) and wicd

I'll update this post once I've got a good solution to setting up network on headless boxes (i.e. a way of bringing up multiple interfaces without knowing their mac addresses)

Original post:

Note: what follows is what I've 'discovered' while learning. If you have suggestions, feel free to post comments. If you have questions, be aware that I'm only a beginning at Arch myself and will probably not have much advice to offer.

The problem: I have a headless server running 32 bit debian. I don't actually use it much, and so I figured I might as well use it to explore Gentoo or Arch. Given that it's a single core atom I didn't want to risk spending hours compiling things just to get things up and running (Feb/March are busy days at Australian Unis) and I figured that Arch, with it's version of ports, ABS, is a better choice for now.

I'm also curious about systemd, which arch uses by default. (that curiosity has currently turned to frustration)

Why Arch?
I do like my debian, but you need to challenge yourself every now and again. I can also see how a more 'sparse' distro might be a good candidate for my cluster nodes -- less overhead is a good thing. But before doing that I need a test case.

Also, even though I've been using debian I occasionally pay a visit to the Arch wiki and forum when I run into trouble with software that I've compiled myself -- Arch is a lot more bleeding edge than debian (which emphasizes stability), and the information there is often quite good.

Well then...
The easiest way (needing little preparation) by far to install arch would be to use ArchBang which is fast, pretty and functional. I don't need the graphical environment, and I want the learning experience. Otherwise I'd definitely give that a look as well: http://archbang.org/

We'll follow this guide: https://wiki.archlinux.org/index.php/Install_from_Existing_Linux

We'll use method "2: Chroot into LiveCD image".

I've ripped out the harddrive from the headless box and have attached it via USB to my main desktop, which runs Debian Testing.

I've rewritten this guide a couple of time -- not everything worked smoothly from the beginning -- so if it's unreadable and confusing, let me know. Also, due to the lack of versioning in blogspot I have had to try to recover/rewrite from memory, which annoys me.

A word of warning: at one point when working in a chroot I accidentally did grub-install /dev/sda instead of /dev/sdc. I then did update-grub in the main system and rebooted. I should've done grub-install /dev/sda again but from the host system. Anyway, you might screw things up, so a first precaution will be to burn a live CD of some linux distro that you can use to rescue your system with if you mess things up too badly. I'm a debian guy, but I found that the Ubuntu Rescue Remix (http://ubuntu-rescue-remix.org/) was a pretty good thing to have lying around.

Overview:
1. Nuke the existing file system (not necessary, but easier)
2. Get the iso and unsquash it
3. Mount the unsquashed image using chroot, and mount your hdd under it. Get packages
4. Chroot inside the chroot and install the bootloader
5. Set up network


Get started


Format/Prepare your harddrive:
In this case I'm using a 40 Gb hard drive. Since it's so small I'll just partition it to hold one / partition and one /swap partition. We'll make the swap partition 1 Gb which should be plenty, given that the box has 512 Mb RAM. Make sure that you leave enough space before you partition -- 1 Mb is overkill, but is something you can afford. In my original attempt I used palimpsest to format my drive and ended up having to shrink the partition using gparted. Better to use fdisk which may be slightly more challenging, but will give you full control.

Remove the harddrive from your target box, and attach it to your running linux desktop e.g. via USB (or by opening the case and hooking it up via sata/pata). If it automounted, do df -h so see what device it is (here /dev/sdc), otherwise just do ls /dev/sd* before and after it's attached.

df -h
Filesystem Size Used Avail Use% Mounted on /dev/sdc1 36G 15G 20G 43% /media/steelhead
Unmount it:
sudo umount /media/steelhead

We now know that our disk is /dev/sdc, so let's get cracking with fdisk
fdisk -u -l /dev/sdc
Disk /dev/sdc: 40 GB, 40007761920 bytes 255 heads, 63 sectors/track, 4864 cylinders, total 78140160 sectors Units = sectors of 1 * 512 = 512 bytes Device Boot Start End Blocks Id System /dev/sdc1 * 63 78140159 39070048 83 Linux
We want our swap to be 1 Gb. (40007761920 bytes /4864 cylinders)/(1024*1024*1024)=.00766038894653320312 Gb per cylinder or ca 130.5 cylinders per Gb. Our first partition will start at cylinder 2 and run until cylinder (4864-131=)4733.

sudo fdisk /dev/sdc
Command (m for help): d
Partition number (1-1): 1
Command (m for help): n
Partition type e extended p primary partition (1-4)
p First cylinder (default 0cyl): 2 Last cylinder or +size or +sizeMB or +sizeKB (default 4863cyl): 4733 Command (m for help): n
Partition type e extended p primary partition (1-4)
p First cylinder (default 0cyl): 4735 Last cylinder or +size or +sizeMB or +sizeKB (default 4863cyl): Command (m for help): w sudo fdisk /dev/sdc Command (m for help): p
Disk /dev/sdc: 40 GB, 40007761920 bytes 255 heads, 63 sectors/track, 4864 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sdc1 2 4734 38025792 83 Linux Warning: Partition 1 does not end on cylinder boundary. /dev/sdc2 4735 4864 1036192 83 Linux
Command (m for help): a Partition number (1-2): 1 Command (m for help): t Partition number (1-2): 2 Hex code (type L to list codes): 82 Changed type of partition 2 to 82 (Linux swap) Command (m for help): w

The cylinder boundary warning is why we started the next partition on cylinder 4735, so as to avoid overlap. 'a' makes the first partition bootable. 't' creates the swap fs.

Create the file system for partition 1:
sudo mkfs.ext4 /dev/sdc1
sudo mkswap /dev/sdc2


Get the iso:
sudo apt-get install bittorrent
cd ~/Downloads
btdownloadcurses https://www.archlinux.org/iso/2013.01.04/archlinux-2013.01.04-dual.iso.torrent

If you're company/isp/whatever has a blanket ban on P2P (e.g. bittorrent) download the file using regular http, e.g.
cd ~/Downloads
wget http://mirror.aarnet.edu.au/pub/archlinux/iso/2013.01.04/archlinux-2013.01.04-dual.iso

However you downloaded it, do:
sudo apt-get install squashfs-tools
sudo mount -o loop archlinux-2013.01.04-dual.iso /mnt
unsquashfs -d /tmp/squashfs-root /mnt/arch/i686/root-image.fs.sfs

This is for a 32 bit install. Most likely you'll want the 64 bit, so change i686 to x86_64 for that. Also note that the unsquashed image is 1.4 Gb so make sure you put it somewhere with sufficient space.

Continue:

sudo umount /mnt
sudo mount -o loop /tmp/squashfs-root/root-image.fs /mnt
sudo mount -t proc none /mnt/proc
sudo mount -t sysfs none /mnt/sys
sudo mount -o bind /dev /mnt/dev
sudo mount -o bind /dev/pts /mnt/dev/pts
sudo cp /etc/resolv.conf /mnt/etc/resolv.conf
sudo chroot /mnt
[root@beryllium /]#

You're now in the chroot.

In the chroot:
mount /dev/sdc1 /mnt
mkdir /run/shm

Since I am installing 32 bit Arch from a 64 bit host, I had to edit /etc/pacman.conf using nano and change
23 Architecture = auto
to
23 Architecture = i686

If you are installing 64 bit Arch from a 64 bit host you don't need to edit anything. Also, the only editor installed by default is nano, not vi, for some reason.

Finally, edit /etc/pacman.d/mirrors and copy/paste (ctrl+k, ctrl+u) the nearest/most logical mirror to the beginning of the file.

pacman-key --init
pacman-key --populate archlinux
pacstrap /mnt base base-devel vim grub-bios openssh
==> Creating install root at /mnt ==> Installing packages to /mnt warning: database file for 'core' does not exist warning: database file for 'extra' does not exist warning: database file for 'community' does not exist :: Synchronizing package databases... core 105.1 KiB 15.8K/s 00:07 [##################] 100% [..] Total Download Size: 163.59 MiB Total Installed Size: 603.01 MiB [..]
I got a couple of errors above re file systems (during grub init) that arch had no business looking at. I don't think it matters since the next chroot is what's important.
pacman -Syy
genfstab -p /mnt >> /mnt/etc/fstab
cat /mnt/etc/fstab
# # /etc/fstab: static file system information # #

# UUID=ae73518f-5675-486e-8cdd-6579c528ab72 LABEL=steelhead /dev/sdc1 / ext4 rw,relatime,data=ordered 0 1 # UUID=f25a484c-e8e2-4ead-808b-96dd5e915020 /dev/sda5 none swap defaults 0 0
Two problems here:
* Since this will be the only hdd on the new system the root partition is probably /dev/sda1. Change the sdc1 to sda1 above OR use UUID
* Likewise, the swap will be the current sdc2 but the future sda2. Alternatively, get the UUID via blkid.

Here's my new two-line version of fstab with uuid:
UUID=ae73518f-5675-486e-8cdd-6579c528ab72 / ext4 rw,relatime,data=ordered 0 1 UUID=a12b484c-f2d2-4edd-8d8b-76ee6f725020 none swap defaults 0 0
Continue:

arch-chroot /mnt

which gives
sh-4.2#

You're now in a chroot inside a chroot.

A chroot within a chroot:


pacman -Syy
pacman-key --init
pacman-key --populate archlinux
ln -s /usr/share/zoneinfo/Australia/Melbourne /etc/localtime
echo "kookaburra" > /etc/hostname

Kookaburra is the hostname of the new Arch box.

Edit /etc/pacman.conf and change Architecture from auto to i686 if you need to.

Edit /etc/locale.gen and uncomment the locales you want. For me
en_AU.UTF-8 UTF-8 en_GB.UTF-8 UTF-8 en_US.UTF-8 UTF-8

Continue working on boot:
locale-gen
echo 'LANG="en_AU.UTF-8"'>/etc/locale
echo 'KEYMAP=us'> /etc/vconsole.conf
mkinitcpio -p linux
==> Building image from preset: 'default' -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img ==> Starting build: 3.7.4-1-ARCH -> Running build hook: [base] [..] ==> Creating gzip initcpio image: /boot/initramfs-linux-fallback.img ==> Image generation successful
grub-mkconfig -o /boot/grub/grub.cfg
Generating grub.cfg ... Found linux image: /boot/vmlinuz-linux Found initrd image: /boot/initramfs-linux.img done

Check /boot/grub/grub.cfg to see that it exists and that it doesn't look like it's about to blow up. Double-check the uuid vs fstab. Check the 'root'. Mine was wrong. I changed it in /boot/grub/grub.cfg by hand, which isn't recommended but we'll deal with that later. In /boot/grub/grub.cfg change hdX to whatever the correct setting is (counting starts at 0) e.g.
  
        set root='hd0,msdos1'

since /dev/sdc in the future will be the only hdd. Changing this file by hand is untenable in the long run, but our goal is to boot and then regenerate the grub.cfg at a later stage.

grub-install /dev/sdc
Installation finished. No error reported.
Then continue -- in this case we're working with a headless box so we want openssh-server and all that:

passwd
Enter new UNIX password: Retype new UNIX password: passwd: password updated successfully
pacman -S net-tools ifplugd dialog sudo wireless_tools wpa_supplicant wpa_actiond systemctl enable sshd.service systemctl enable net-auto-wired.service systemctl enable net-auto-wireless.service cp /etc/network.d/examples/ethernet-dhcp /etc/network.d/ethli

Edit the /etc/network.d/ethli file as shown in the next section.(the ethli name holds no significance)

adduser verahill -m

Edit /etc/sudoers and add:
verahill ALL=(ALL) ALL
pacman -S pkgtools mlocate htop screen elinks

Time to exit:

sh-4.2# exit
exit
[root@beryllium /]# umount /mnt
[root@beryllium /]# exit

You're now back in your normal host filesystem, and unmount the rest:
sudo umount /mnt/proc
sudo umount /mnt/sys
sudo umount /mnt/dev/pts
sudo umount /mnt/dev
sudo umount /mnt

At this point you have a bootable system. I attached the hdd to my laptop using USB, and chose to boot from it. Went fine. However, it turns out that networking in arch and/or using systemd is very different from debian and initd.

Networking:
Networking should be one of the easiest things in the world to set up, since if you have network access everything else will eventually sort itself out. No network means no remote access via ssh if the screen/keyboard/mouse get screwed up. No network means that you can't install anything. No network means that you can't go online and look up solutions.

initd was easy -- you checked your udev rules, then edited /etc/network/interfaces and added an auto eth0 line followed by e.g. iface eth0 inet dhcp or a more extensive static definition. It was easy.
So I'm a bit frustrated that systemd seems to make the whole process of managing network interfaces so much more difficult. At least for a newcomer.

With systemd your interfaces may not have simple names like eth0 right off the bat, but may have names like enpS08 instead -- it does make things a bit more difficult and unpredictable -- remember that my goal is to get a headless box up and running and that I have no way of getting any error messages -- openssh will simply have to work from the start. I mean, there's a certain logic to systemd, but there are plenty of frustrating issues with it too if you're used to chucking start-up commands in rc.local and setting up your own network devices.

Some of this stuff you could do by hand on a running system, but that's not very helpful when you are working with a headless box where your only option is to connect via ssh (technically I could do rs-232 but I can't find a cable).

Anyway.

Create /etc/udev/10-persisten-net.rules
SUBSYSTEM=="net", ATTR{address}=="00:2e:9e:2e:bb:20", KERNEL=="eth*", NAME="ethli"
SUBSYSTEM=="net", ATTR{address}=="00:e2:bb:be:d4:c8", KERNEL=="wlan*", NAME="wlan0"
and put the mac addresses of you interfaces in it.

Edit /etc/conf.d/netcfg to use the correct WIRED_INTERFACE,
NETWORKS=(ethli ethkb5 ethkb3 wpa-wireless)
WIRED_INTERFACE="ethli"
WIRELESS_INTERFACE="wlan0"
ethkb5, ethb3 and wpa-wireless are other files that I've set up in /etc/network.d/, although at this point they are pretty darned useless -- I can define plenty of profiles, but I can only define a single WIRED_INTERFACE from what I can tell. So only one interface will be started by netcfg.

Copy /etc/network.d/example/ethernet-dhcp to /etc/network.d/ethli, and edit the interface name.
CONNECTION='ethernet'
DESCRIPTION='A basic dhcp ethernet connection using iproute'
INTERFACE='ethli'
IP='dhcp'
Do the same for any other profiles.
For wireless, all you need to do is copy  /etc/network.d/examples/wpa-wireless to /etc/network.d/ and add the password in clear text (i.e. no need to use wpa_password). Change the permissions so that it's 'safe'.

Here's my wpa-wireless
CONNECTION='wireless'
DESCRIPTION='A simple WPA encrypted wireless connection'
INTERFACE='wlan0'
SECURITY='wpa'
ESSID='verahillunwired'
KEY='sadfUsfdjdsfH87j'
IP='dhcp'

The wired network should now start on boot. To launch it manually, just do
sudo netcfg ethli

This is where I'm currently at:
* arch works fine when I boot it by attaching the hdd via usb to a laptop or desktop and booting off of it. I have working internet and the sky's the limit to what can be done

* I can't get it to boot my headless box (or at least not to get a working network connection), and because it's completely headless (it's a server with only eth ports and an RS 232 port) I have no idea why. The logging in systemd leaves a lot to be desired it seems. I'm waiting for an rs-232 cable to arrive via ebay, so we'll see.
The two possibilities that I'm entertaining right now is that either there's an issue with the network devices or...well, I don't know. I'm also getting tired of netcfg and will probably switch to wicd -- it's not quite what I want, but at least I know that it works.
But more about that some other time...

Error 1:
# pacstrap /mnt
==> Creating install root at /mnt
mount: mount point /mnt/dev/shm is a symbolic link to nowhere
==> ERROR: failed to setup API filesystems in new root

Solution:
mkdir /run/shm

On debian /dev/shm points to /run/shm via a symlink, so your chroot has a dead link.

Error 2:
error: key "E62F853100F0D0F0" could not be looked up remotely
error: psmisc: key "FCF2CB179205AC90" is unknown
error: key "FCF2CB179205AC90" could not be looked up remotely
error: reiserfsprogs: key "7F2D434B9741E8AC" is unknown
error: key "7F2D434B9741E8AC" could not be looked up remotely

Solution:
pacman-key --init
pacman-key --populate archlinux

You need to get all the gpg keys so you can check the package signatures.

Error 3:
warning: database file for 'core' does not exist
warning: database file for 'extra' does not exist
warning: database file for 'community' does not exist

Solution:
pacman -Syy

Not sure, but probably due to me interrupting pacman rather rudely with ^C at some point.

Error 4:
# arch-chroot /mnt pacman -S grub-bios
error: failed to prepare transaction (package architecture is not valid) :: package grub-bios-2.00-1-i686 does not have a valid architecture
Solution:
pacstrap /mnt grub-bios

And then skip the arch-root command

Error:
grub-install /dev/sdc
/usr/sbin/grub-bios-setup: warning: your embedding area is unusually small.  core.img won't fit in it..
/usr/sbin/grub-bios-setup: warning: Embedding is not possible.  GRUB can only be installed in this setup by using blocklists.  However, blocklists are UNRELIABLE and their use is discouraged..
/usr/sbin/grub-bios-setup: error: will not proceed with blocklists.

Solution: You partitioned your system without leaving enough space before the first partition. In my case I had 32 sectors*512 bytes.
sudo apt-get install gparted
gparted
Start gparted and shrink the partition. I put 2 Mb (probably overkill) of free space in front, and aligned to cylinder.

28 January 2013

328. Liberate your router: dd-wrt on Netgear WGT624 v4

UPDATE 1 Feb 2013: I haven't had any explicit problems with my router since flashing it. Everything is apparently working well and my network connection is reliable and fine (if only subjectively a bit slower than before --  running a speed test shows that it's as fast as ever so not sure what's happening). HOWEVER, I've suddenly started having issues with ECCE and submitting jobs via a frontendMachine -- I kept getting "cannot 'cd' to run directory" errors, but the ECCE log contains no errors messages at all. This wouldn't happen for very small NWChem input files, and it would happen ca 80% of the time. Normally I wouldn't suspect this was a router issue, but changing back to my (unflashed) AR430W resolved the issues immediately. Somehow I suspect this is a router version of this http://verahill.blogspot.com.au/2012/09/briefly-packet-corrupt-during-ssh.html, but then I should see error messages in the ECCE log...

Other than that I'm really happy with dd-wrt (no sarcasm intended -- I've had no other issues and I love the power dd-wrt gives me over my hardware).

Finally, there's the old adage about correlation vs causuality. We'll see if the errors start popping up again while using my AR430W.

Original post:

I've been using Tomato with my WRT54G for a couple of years now, and I'm incredibly happy with it. Since I have a couple of old routers (airlink 1010 ar430w and netgear wgt624 v4) with stock firmware lying around I figured it was time to turn them into something useful. So here's how to flash the netgear router. If it stands up to sustained use I'll be writing an AR430W guide later.


dd-wrt


Lengthy preamble
The stock firmware basically does nothing for me -- it's clunky, slow, and there's no terminal access. In particular, I want busybox/ssh, Tomato does all that for me, but it doesn't support a particularly wide range of routers (I reckon that Tomato is the reason why Linksys WRT54GL still costs $90 in Australia, in spite of being old as sin -- those who doubt the value of opening up their hardware may want to consider the RoI on that one)

In addition to Tomato, there's also DD-WRT (supported devices) and OpenWRT (supported devices). DD-WRT support a huge number of routers, but it appears to be a whole lot more complicated to install than Tomato. Maybe this varies according to the router as well.

For instructions you're referred via the database to the dd-wrt forum thread about your router. The problem with this is that you'll be facing 30-odd pages with instructions, problems, dead-ends etc. Some threads end with a step-by-step summary on how to install dd-wrt, but not all do.

Anyway, here's my best attempt at writing a simple and complete step-by-step guide to replacing the stock firmware on Netgear WGT24 v4 with DD-WRT on Debian Testing/Wheezy. I'm basically just following this blog post: http://lauriaus.no-ip.org/blog/?p=90 , but hopefully I've added enough detail to make it possible for just about anyone to follow this guide.

Please consult http://www.dd-wrt.com/site/support/router-database to see what files you need. NOTE: the files below only apply to v4 of Netgear WGT624. Installing them on any other router may brick it.



On your linux computer:

Get the files:
sudo apt-get install atftpd tftp putty
cd /tmp
mkdir ftpdboot
cd ftpdboot/
wget http://www.dd-wrt.com/dd-wrtv2/downloads/others/redboot_collection/images_default/redboot_ap61_16M_4M_admtek.rom
wget http://www.dd-wrt.com/routerdb/de/download/Netgear/WGT624/v4/linux.bin/3614 -O linux.bin
wget http://www.dd-wrt.com/routerdb/de/download/Netgear/WGT624/v4/wgt624v4-firmware.bin/3613 -O wgt624v4-firmware.bin

Edit /etc/default/atftpd:
USE_INETD=false #true OPTIONS="--tftpd-timeout 300 --retry-timeout 5 --mcast-port 1758 --mcast-addr 239.239.239.0-255 --mcast-ttl 1 --maxthread 100 --verbose=5 /tmp/ftpdboot"

Edit /etc/inetd.conf
32 tftp dgram udp4 wait nobody /usr/sbin/tcpd /usr/sbin/in.tftpd --tftpd-timeout 300 --retry-timeout 5 --mcast-port 1758 --mcast-addr 239.239.239.0-255 --mcast-ttl 1 --maxthread 100 --verbose=5 /tmp/ftpdboot
and do
sudo /etc/init.d/openbsd-inetd reload

for good luck. If you don't have openbsd-inetd you may have xinetd or inetutils-inetd installed instead (I think openbsd-inetd is default on debian). Edit the command as necessary.

Edit your /etc/network/interfaces file:
auto eth0 iface eth0 inet static address 192.168.1.155 gateway 192.168.1.1 netmask 255.255.255.0

and run
sudo service networking restart

Make sure that your card came up ok (do e.g. ip addr)
2: eth0: broadcast mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:26:9e:27:9b:20 brd ff:ff:ff:ff:ff:ff inet 192.168.1.155/24 brd 192.168.1.255 scope global eth0
Continue.

Prepare two terminals, side by side (or start a screen session with two tabs open). In one, type
echo "^C"> end.txt
putty telnet 192.168.1.1:9000 -m end.txt

But don't hit enter after the second command.

In the other terminal, type
ping 192.168.1.1

but don't hit enter.

Connect the ethernet port on your computer to one of the ethernet LAN ports (not WAN/Internet) on your router.

You are next going to unplug the power from the router, and hit enter after the ping command. Immediately when you get ping replies:
64 bytes from 192.168.1.1: icmp_req=4 ttl=64 time=0.371 ms
you hit enter after the putty command in the other window. If nothing good happens, then redo (i.e. unplug the router, hit enter after the ping command etc. Don't start the ping until you're re-plugged the router).

Ready? GO!
ping 192.168.1.1
64 bytes from 192.168.1.1: icmp_req=1 ttl=64 time=0.371 ms
putty telnet 192.168.1.1:9000 -m end.txt

And you should get

Before you continue make sure that you've opened up your firewall e.g. if you're not connected to the internet you can go crazy like this:
sudo iptables -P INPUT ACCEPT
sudo iptables -P OUTPUT ACCEPT
sudo iptables -P FORWARD ACCEPT

And don't forget to restore your firewall once you're done.
Time to get dangerous.

RedBoot> fis init
About to initialize [format] FLASH image system - continue (y/n)? Y *** Initialize FLASH Image System ... Erase from 0xbffe0000-0xbfff0000: . ... Program from 0x80ff0000-0x81000000 at 0xbffe0000: . RedBoot> ip_address -h 192.168.1.155 IP: 192.168.1.1/255.255.255.0, Gateway: 192.168.1.254 Default server: 192.168.1.155
RedBoot> load -r -b %{FREEMEMLO} redboot_ap61_16M_4M_admtek.rom
Using default protocol (TFTP) TFTP timed out 1/15 Can't load 'redboot_ap61_16M_4M_admtek.rom': operation timed out
Try again:
RedBoot> load -r -b %{FREEMEMLO} redboot_ap61_16M_4M_admtek.rom
Using default protocol (TFTP) Raw file loaded 0x80040c00-0x8005007f, assumed entry at 0x80040c00
RedBoot> fis create -l 0x30000 -e 0xbfc00000 RedBoot fis create -l 0x30000 -e 0xbfc00000 RedBoot An image named 'RedBoot' exists - continue (y/n)? y ... Erase from 0xbfc00000-0xbfc30000: ... ... Program from 0x80040c00-0x80050080 at 0xbfc00000: . ... Erase from 0xbffe0000-0xbfff0000: . ... Program from 0x80ff0000-0x81000000 at 0xbffe0000: .
RedBoot> reset

You'll see a couple of flashing lights on the router as the only indication that something just happened. Kill your current putty connection and start a new one.
=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2013.01.28 20:29:38 =~=~=~=~=~=~=~ ^C
RedBoot> fis init
About to initialize [format] FLASH image system - continue (y/n)? y *** Initialize FLASH Image System ... Erase from 0xbffe0000-0xbfff0000: . ... Program from 0x80ff0000-0x81000000 at 0xbffe0000: .
RedBoot> ip_address -h 192.168.1.155
IP: 192.168.1.1, Default server: 192.168.1.155
RedBoot> load -r -b 0x80041000 linux.bin
Using default protocol (TFTP) Raw file loaded 0x80041000-0x803ecfff, assumed entry at 0x80041000
RedBoot> fis create linux

Be patient -- this step takes a long time: 19 minutes in my case (some routers take an hour). Write down the time when it starts and WAIT at least 20 minutes.
... Erase from 0xbfc10000-0xbffbc000: ........................................................... ... Program from 0x80041000-0x803ed000 at 0xbfc10000: ........................................................... ... Erase from 0xbffe0000-0xbfff0000: . ... Program from 0x80ff0000-0x81000000 at 0xbffe0000: .
RedBoot> fconfig
Run script at boot: true Boot script: .. fis load -l kernel .. go Enter script, terminate with empty line
>> fis load -l linux >> exec >> Boot script timeout (1000ms resolution): 12 Use BOOTP for network configuration: false bootp_my_gateway_ip: 192.168.1.254 Local IP address: 192.168.1.1 bootp_my_ip_mask: 255.255.255.0 Default server IP address: 192.168.1.55 Console baud rate: 9600 GDB connection port: 9000 Force console for special debug messages: false net_debug: false Update RedBoot non-volatile configuration - continue (y/n)? y ... Erase from 0xbffe0000-0xbfff0000: . ... Program from 0x80ff0000-0x81000000 at 0xbffe0000: . RedBoot> reset

Done!

You can now navigate to 192.168.1.1 in your router, but unplug, replug the router for good luck.
Success!


I created a user called admin and set a password i.e. there's no pw or username you need to know a priori.
click on services


check sshd

Don't trust important infrastructure with passwords. Use keys.




And finally
ssh root@192.168.1.1

and hopefully you're in.

First impressions:
The busybox ('linux') version is a bit too sparse for my liking -- no netstat command...but it's still obviously a major step up from the stock firmware. dd-wrt is different from tomato -- if you're used to one you're not necessarily going to feel comfortable with the other. Luckily, dd-wrt is widely used and there are plenty of resource online. In addition, there's a demo ( http://www.dd-wrt.com/demo/ ) so you can try it out before installing it.

How to set up 'static' dhcp (i.e. make sure that some computers always have the same IP address while still running a dhcp server) wasn't completely obvious either, but this post helped: http://www.dd-wrt.com/wiki/index.php/Static_DHCP


24 January 2013

326. Compiling Thunderbird 17.0.2 on Debian Testing

I tested this compile in a bare chroot, so the dependencies should be pretty much hammered out. The build is easy but fairly slow.

sudo apt-get install bzip2 build-essential python zip libgtk2.0-dev libdbus-glib-1-dev libasound2-dev libogg-dev libxt-dev yasm libcurl4-openssl-dev mesa-common-dev
mkdir ~/tmp
cd ~/tmp
wget ftp://ftp.mozilla.org/pub/mozilla.org/thunderbird/releases/17.0.2/source/thunderbird-17.0.2.source.tar.bz2
rm -rf comm-release/
tar xvf thunderbird-17.0.2.source.tar.bz2
mkdir bldthunder17
cd bldthunder17/
../comm-release/./configure --disable-necko-wifi
make
sudo make install

That took 122 minutes on a single core.
Checkinstall doesn't work and ends with segfault.


23 January 2013

325. Compiling ECCE 6.4 on Debian Testing

!NOTE! If you provide ECCE with 'localhost' as the hostname, be aware that this will block outside access: http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id858/#post_3178

There's a new release of ECCE 6.4 out which fixes the following bug: http://verahill.blogspot.com.au/2012/11/minor-bug-in-ecce.html


A note on Java: I've only had luck with the openjdk packages -- not with the Oracle/sun java ones (see here: http://verahill.blogspot.com.au/2012/06/building-ecce-on-debian-testingwheezy.html).

This build was tested in a chrooted Testing/Wheezy i.e. I should've caught most of the necessary packages.


Compiling:
Download the source to ecce from http://ecce.emsl.pnl.gov/using/download.shtml, and put it in e.g. ~/tmp

sudo apt-get install bzip2 build-essential autoconf libtool ant pkg-config gtk+-2.0-dev libxt-dev csh gfortran openjdk-7-jdk python-dev libjpeg-dev imagemagick xterm
cd ~/tmp
tar xvf ecce-v6.4-src.tar.bz2
cd ecce-v6.4/
export ECCE_HOME=`pwd`
cd build/
./build_ecce
Hit return if xterm was found... The /home/sandbox/tmp/ecce-v6.4/scripts/sysdir script identifies the build platform directory as: empty Because this value is empty no platform-specific parent directory will be created for ECCE executables, libraries, etc. This works fine unless your site needs support for multiple platforms. Finished checking prerequisites for building ECCE. Do you want to skip these checks for future build_ecce invocations (y/n)? Y
./build_ecce
Xerces built
./build_ecce
Mesa OpenGL built
./build_ecce
wxWidgets built
./build_ecce
running build_ext wxPython built
./build_ecce
Apache HTTP server built
./build_ecce
Making combined tar file ecce.v6.4.tar Copying NWChem binary distribution nwchem-6.1.1-binary-rhel5-gcc4.1.2-m64.tar.bz2 Copying NWChem common distribution nwchem-6.1.1-binary-common.tar.bz2 Concatenating install script and combined tar file ecce.v6.4.tar create_ecce_bin finished ECCE built and distribution created in /home/sandbox/tmp/ecce-v6.4
cd ../ ./install_ecce.v6.4.csh

And see the next section for the installation steps.
The compilation steps take progressively longer and longer, so be patient during the build.

Installing:
Digital ink is cheap, so I'll show the whole process:
./install_ecce.v6.4.csh
Main ECCE installation menu =========================== 1) Help on main menu options 2) Prerequisite software check 3) Full install 4) Full upgrade 5) Application software install 6) Application software upgrade 7) Server install 8) Server upgrade IMPORTANT: If you are uncertain about any aspect of installing or running ECCE at your site, please refer to the detailed ECCE Installation and Administration Guide at http://ecce.pnl.gov/docs/installation/2864B-Installation.pdf Hit at prompts to accept the default value in brackets. Selection: [1] 3 Host name: [beryllium] localhost Application installation directory: [/home/sandbox/tmp/ecce-v6.4/ecce-v6.4/apps] /home/sandbox/.ecce/apps Server installation directory: [/home/sandbox/.ecce/server] ECCE v6.4 will be installed using the settings: Installation type: [full install] Host name: [localhost] Application installation directory: [/home/sandbox/.ecce/apps] Server installation directory: [/home/sandbox/.ecce/server] Are these choices correct (yes/no/quit)? [yes] Installing ECCE application software in /home/sandbox/.ecce/apps... Extracting application distribution... Extracting NWChem binary distribution... Extracting NWChem common distribution... Extracting client WebHelp distribution... Configuring application software... Configuring NWChem... Installing ECCE server in /home/sandbox/.ecce/server... Extracting data server in /home/sandbox/.ecce/server/httpd... Extracting data libraries in /home/sandbox/.ecce/server/data... Extracting Java Messaging Server in /home/sandbox/.ecce/server/activemq... Configuring ECCE server... ECCE installation succeeded. *************************************************************** !! You MUST perform the following steps in order to use ECCE !! -- Unless only the user 'sandbox' will be running ECCE, start the ECCE server as 'sandbox' with: /home/sandbox/.ecce/server/ecce-admin/start_ecce_server -- To register machines to run computational codes, please see the installation and compute resource registration manuals at http://ecce.pnl.gov/using/installguide.shtml -- Before running ECCE each user must source an environment setup script. For csh/tcsh users add this to ~/.cshrc: if ( -e /home/sandbox/.ecce/apps/scripts/runtime_setup ) then source /home/sandbox/.ecce/apps/scripts/runtime_setup endif For sh/bash users, add this to ~/.profile or ~/.bashrc: if [ -e /home/sandbox/.ecce/apps/scripts/runtime_setup.sh ]; then . /home/sandbox/.ecce/apps/scripts/runtime_setup.sh fi ***************************************************************

Instead of following the instructions above I normally do:
echo 'export ECCE_HOME=/home/sandbox/.ecce/apps' >> ~/.bashrc
echo 'PATH=$PATH:/home/sandbox/.ecce/server/ecce-admin/:/home/sandbox/.ecce/apps/scripts/' >> ~/.bashrc
echo 
source ~/.bashrc

You can now start ecce by either doing
ecce

or if that complains, do
start_ecce_server

then waiting a little while (10 s), followed by
ecce


Apppendix:

Selecting Java version
sudo update-alternatives --config java
There are 7 choices for the alternative java (providing /usr/bin/java). Selection Path Priority Status ------------------------------------------------------------ 0 /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java 1061 auto mode 1 /usr/bin/gij-4.4 1044 manual mode 2 /usr/bin/gij-4.6 1046 manual mode 3 /usr/bin/gij-4.7 1047 manual mode 4 /usr/lib/jvm/j2re1.6-oracle/bin/java 314 manual mode 5 /usr/lib/jvm/j2sdk1.6-oracle/jre/bin/java 315 manual mode 6 /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java 1061 manual mode * 7 /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java 1051 manual mode

I've got ECCE running fine with openjdk 7 as well as openjdk 6.

22 January 2013

324. Setting up a private git server

Update 11 Feb 2013: fixed a typo that ruined everything.

I've recently moved one of my more mature projects to sourceforge.net, which allowed me to re-discover the joys of using a version management system, in this case git. While I've spent some time playing with SVN in the past, git feels more natural to me. That's it's used widely in the FOSS community certainly doesn't hurt either.

So that got me interested in setting up a local git server for projects that may or may not result in anything tangible i.e. projects that I don't want to put online, but are under so much flux that they may break anytime (and I thus want to be able to roll back).

[A pointer to my students: you can obviously use git to maintain versions of your .tex documents as well...]

In this particular case the development machine and the git host are the same. The project is called shellnmr (an attempt to make a bash-like shell in python for NMR processing)

This is very much inspired by/stolen from: http://tumblr.intranation.com/post/766290565/how-set-up-your-own-private-git-server-linux


Getting set up:

On the git host (which has not yet got a copy of the source code)
sudo apt-get install git
mkdir ~/work/programming/var -p
cd ~/work/programming/var
mkdir shellnmr
cd shellnmr/
git init --bare
Initialized empty Git repository in /home/me/work/programming/var/shellnmr
First time:
I''ll use localhost below because in this particular example the git host and the development machine is the same.

On the/a development machine, where we keep the already existing project in ~/work/shellnmr


cd ~/work/shellnmr
git init
Initialized empty Git repository in /home/me/work/shellnmr/.git/
git add . git commit -m 'initial commit'
[master (root-commit) 5214860] initial commit 2 files changed, 178967 insertions(+) create mode 100644 functions.py create mode 100755 main.py
git remote rm origin git remote add origin me@localhost:/home/me/work/programming/var/shellnmr git push origin master
Counting objects: 2, done. Delta compression using up to 3 threads. Compressing objects: 100% (18/18), done. Writing objects: 100% (2/2), 339.14 KiB, done. Total 2 (delta 5), reused 0 (delta 0) To me@localhost:/home/me/work/programming/var/shellnmr * [new branch] master -> master


Everyday usage:

Say you've edited the code on your development machine and want to commit the changes:
git commit -a
git push

Easy.

Let's say you go home and want to continue editing the code there and that this is the first time you're doing so:
git clone user@githost:/home/me/work/programming/var/nmrshell nmrshell-code

When you're done editing, just commit as normal;
git commit -a
git push

Let's say you've edited the code on another computer, committed the code, and then want to get the latest updates on your regular development machine again:
git pull origin master

And that's about it.

GUI - gitk:
The debian repos have a git GUI as well. To install do
sudo apt-get install git-gui gitk

Start it by launching gitk from the directory where you keep your files (on the development machine)

18 January 2013

322. Libreoffice and zotero for reference management

I'm happy using latex and bibtex whenever possible, and I'm forced to use Microsoft Office and Endnote in most of my collaborations, so I;m not that interested in libreoffice/openoffice.

However, having heard of a reference manager called Zotero has made me curious to learn more.

You can either get it as a plug-in for firefox, or as a stand-alone program which plugs into any browser of your choice.

While I haven't used Endnote seriously for about half a decade and thus may not know about any fancy recently added functionality, I just can't see any reason whatsoever to use Endnote anymore if you have a choice. Obviously, given the inertia of the 'common user' and the tendency for people to equal costly with good and cheap with rubbish, it will probably take a generation or two for change to happen, no matter how good zotero gets.

And here's another thing: when I went to Google Scholar using Chrome, zotero was recognised, and a list of 'updates' was suggested. I wasn't really interested, until I had a look at them -- Scholar had generated a pretty impressive list of articles that it thought might interest me. The match was surprisingly good. If nothing else I'll certainly make sure to keep my zotero database synced with my bibtex one.


Get it:
sudo apt-get install bzip2
wget http://download.zotero.org/standalone/3.0.11.1/Zotero-3.0.11.1_linux-x86_64.tar.bz2
tar xvf Zotero-3.0.11.1_linux-x86_64.tar.bz2
cd Zotero_linux-x86_64/
./run-zotero.sh

First time:
First time you run it it opens a browser window as well as an installation process
For some reason I keep getting errors about DBus even though the dbus-daemon is running. It hasn't caused any other problems beyond the error message in the screenshot.
Even the openoffice I installed for testing purposes got detected

And you're done

To make it work in Chrome, install the extension

The main Zotero window, without any references


Testing:
Importing a bibtex library

Everything looks ok


If you click on the icon by the arrow you can insert a citation. You need to have zotero running in the background though.
Determine what format to use

Generate the bibliography and you're done



Google Scholar, Chrome and Zotero play nicely together



16 January 2013

321. Compiling Kernel 3.7.3 (and 3.7.2) on Debian Testing/Wheezy. More data on make -jN.

Updated for 3.7.3

Since post '319. Collection of errors when compiling kernel 3.7.x on AMD FX 8150' is getting traffic from people wanting to compile kernel 3.7.2, and because I didn't know whether the azx_runtime_suspend bug had been fixed, I had to try it out. So here's how to compile kernel 3.7.2 and 3.7.3 -- for 3.7.2 simply replace all instances of 3.7.3..

Looking at the code changes here: http://lists-archives.com/linux-kernel/27763782-alsa-hda-move-runtime-pm-check-to-runtime_idle-callback.html and comparing with what I'm actually seeing in sound/pci/hda/hda_intel.c it seems that 3.7.2 and 3.7.3 have been fixed and no patches need to be applied.

Testing the kernel bears that out.


Compiling the kernel
sudo apt-get install kernel-package fakeroot build-essential ncurses-bin ncurses-dev
mkdir ~/tmp
cd ~/tmp
wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.7.3.tar.bz2
tar xvf linux-3.7.3.tar.bz2
cd linux-3.7.3/
cat /boot/config-`uname -r`>.config
make oldconfig
make-kpkg clean

If you want to add specific drivers etc to the kernel, run
make menuconfig

Note that if you're transitioning from kernel 3.5 to 3.7 you will needto specifically and explicitly include a lot of the graphics (pci tv cards, usb web cams) drivers that used to be automatically included before. Then continue:

time fakeroot make-kpkg -j6 --initrd kernel_image kernel_headers
sudo dpkg -i ../linux-image-3.7.3_3.7.3-10.00.Custom_amd64.deb ../linux-headers-3.7.3_3.7.3-10.00.Custom_amd64.deb

And you're done. Keep reading to learn more about -j6.


Optimal -jN
See here for another post on -jN: http://verahill.blogspot.com.au/2013/01/305-make-jn-should-n-equal-number-of.html. In short, it's not always clear whether N should equal the number of cores, or be larger than the number of cores. In that post, N+1 was the optimal configuration, but that was a very short compilation where i/o likely played a large role.

More data is needed, so here it is. Seems like N=number of cores is the best option for long builds (as was pointed out to me in a comment). This was done with kernel 3.7.2.

On a four-core Intel i5-2400 with 16 Gb memory
N Time ------------- 2 30m 58s 3 22m 36s 4 19m 49s 5 22m 2s 6 23m 13s

Acquired using sar/sysstat
Here's what's happening with -j4:
Basically, the first 15 minutes things are running in parallel, with t i/o slowing things down during the last 5 minutes.

On a six-core AMD Phenom II 1055T with 8 Gb memory
N Time (s) ------------- 4 34m 16s 5 27m 19s 6 24m 60s 7 30m 18s 8 31m 47s


Hardware profiles:
Intel machine:
00:00.0 0600: 8086:0100 (rev 09) 00:02.0 0300: 8086:0102 (rev 09) 00:16.0 0780: 8086:1c3a (rev 04) 00:16.3 0700: 8086:1c3d (rev 04) 00:19.0 0200: 8086:1502 (rev 04) 00:1a.0 0c03: 8086:1c2d (rev 04) 00:1b.0 0403: 8086:1c20 (rev 04) 00:1c.0 0604: 8086:1c10 (rev b4) 00:1c.2 0604: 8086:1c14 (rev b4) 00:1d.0 0c03: 8086:1c26 (rev 04) 00:1e.0 0604: 8086:244e (rev a4) 00:1f.0 0601: 8086:1c4e (rev 04) 00:1f.2 0104: 8086:2822 (rev 04) 00:1f.3 0c05: 8086:1c22 (rev 04)

AMD machine:
00:00.0 0600: 1022:9601 00:01.0 0604: 1022:9602 00:07.0 0604: 1022:9607 00:11.0 0106: 1002:4390 00:12.0 0c03: 1002:4397 00:12.1 0c03: 1002:4398 00:12.2 0c03: 1002:4396 00:13.0 0c03: 1002:4397 00:13.1 0c03: 1002:4398 00:13.2 0c03: 1002:4396 00:14.0 0c05: 1002:4385 (rev 3c) 00:14.1 0101: 1002:439c 00:14.2 0403: 1002:4383 00:14.3 0601: 1002:439d 00:14.4 0604: 1002:4384 00:14.5 0c03: 1002:4399 00:18.0 0600: 1022:1200 00:18.1 0600: 1022:1201 00:18.2 0600: 1022:1202 00:18.3 0600: 1022:1203 00:18.4 0600: 1022:1204 01:05.0 0300: 1002:9715 01:05.1 0403: 1002:970f 02:00.0 0200: 10ec:8168 (rev 03) 03:05.0 0200: 10ec:8169 (rev 10

320. Wsearch32 in Wine

Wsearch32 is a windows program that does what it's supposed to -- open mass spectrometer data files. Because our students use it quite a lot and to save me time in training them, here's how to install wsearch32 and how to get started with it. It runs almost perfectly in Wine.

One notable thing about Wsearch is that it opens HP Agilent Chemstation .D files without any problems. It's notable because different versions of HP Agilent Chemstation can't even open files written by other versions (yup, it's true...) of the same damned program.

Another obvious advantage of Wsearch is that it makes it very easy to export data, although it only exports one spectrum at a time (i.e. you can't get the full time-dependent data set).

WSearch was written by Frank Antolasic at the RMIT in Melbourne, Australia.


What doesn't work under Wine: It will freeze if you try to open several files at the same time. This does not happen on native Windows.




Installation
Download it from here: http://www.wsearch.com.au/
wget http://www.wsearch.com.au/wsearch32/wsearch32.zip
unzip wsearch32.zip
and follow the instructions:
There's nothing odd about the installation process

It works as well as under windows


Screenshots
This is the default view

Click on the import icon (the red one in the left corner), and select your file. There are plenty of supported formats.

This is the TIC view. Click anywhere on the TIC to get the corresponding spectrum

TIC and spectrum view.

Click on the M/I icon on the right to get a list of all the data in the spectrum plot..
You can then save the spectrum in a range of ascii based formats.


You can get the acquisition headers under Chromatogram/List header info. You can also make ion plots  in the same menu.
Wsearch also comes with a simple isotopic pattern calculator.

The Help works fine under linux/wine


This is the free version of wsearch -- there is some functionality missing.