Showing posts with label debian. Show all posts
Showing posts with label debian. Show all posts

31 January 2017

633. Multiwfn on debian, with gaussian

Not much to say. I wanted to do some Bader-type AIM analyses, and I don't have AIM2000/AIMPAC. Multiwfn can do this, and a lot more.

Setting up multiwfn
Download the precompiled Multiwfn binaries here:

Extract the binary and the copy the directory to .e.g /opt

$ ls /opt/Multiwfn_3.3.9_bin_Linux/
examples  Multiwfn  settings.ini

Edit your ~/.bashrc and add:
export KMP_STACKSIZE=64000000
export Multiwfnpath=/opt/Multiwfn_3.3.9_bin_Linux

When I hit 0 to visualize something, the plot screen is empty. Check/uncheck a box in the window and the plot will show.
Generating wfx files in gaussian:
If you've already run a job that has generated a .chk file, then run a simple quick job like this:
%nprocshared=2 %mem=500Mb %chk=test.chk #P chkbasis geom=allcheck guess=(check,only) output=wfx pop=(none) test.wfx

To analyse the .wfx file using multiwfn, look at the examples in the manual:
The strength of multiwfn is that it can do a LOT. That's also its weakness, in the sense that the menu options are a bit overwhelming. Following the examples in the manual is probably mandatory.

06 February 2016

629. Google drive with rclone on debian jessie

My free dropbox account has 6Gb storage, whereas my google drive has 15Gb. Dropbox pro is way too expensive, so here's what you can do to get your google drive to work ok-ish with linux.

To set up rclone, do:
cd ~/Downloads
cd rclone-v1.27-linux-amd64
sudo cp rclone /usr/bin/
rclone config
No remotes found - make a new one n) New remote q) Quit config n/q> n name> google_drive What type of source is it? Choose a number from below 1) amazon cloud drive 2) b2 3) drive 4) dropbox 5) google cloud storage 6) swift 7) hubic 8) local 9) onedrive 10) s3 11) yandex type> 3 Google Application Client Id - leave blank normally. client_id> Google Application Client Secret - leave blank normally. client_secret> Remote config Use auto config? * Say Y if not sure * Say N if you are working on a remote or headless machine or Y didn't work y) Yes n) No y/n> y If your browser doesn't open automatically go to the following link: Log in and authorize rclone for access Waiting for code... Got code -------------------- [google_drive] client_id = client_secret = token = {"access_token":"y**5","token_type":"Bearer","refresh_token":"1/o**T","expiry":"2016-02-06T18:41:22.167889627+11:00"} -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y Current remotes: Name Type ==== ==== google_drive drive e) Edit existing remote n) New remote d) Delete remote q) Quit config e/n/d/q> q

Syncing is a bit of an issue as you can't sync based on last changed file. So, my solution is that each device has it's own folder that it syncs up to, and each device syncs down all other folders. For my two devices, nb and be, I have thus set it up like this:

Create new dirs and sync:
mkdir ~/gdrive/nbdrive -p
mkdir ~/gdrive/bedrive

On Nb the crontab looks like this:
*/2 *    * * * rclone sync /home/me/gdrive/nbdrive google_drive:/nbdrive
*/10 *   * * * rclone sync google_drive:/bedrive /home/me/gdrive/bedrive

and on Be it looks like this:
*/10 *    * * * rclone sync google_drive:/nbdrive /home/me/gdrive/nbdrive 
*/2 *   * * * rclone sync /home/me/gdrive/bedrive google_drive:/bedrive 

If I do work on a file on Be I then copy it to bedrive, and edit it. To edit it late on Nb I copy it from bedrive to nbdrive and edit it. Of course I could always sync manually and avoid this, but the above is a solution for people who might not feel comfortable working on the command line or if you want everything automated.

19 January 2016

627. Very briefly: Turn off screen blanking in GNOME/debian jessie

The only thing that worked for turning off screen-blanking when using youtube or vlc was an ugly little hack:
I installed xdotool and ran crontab -e and added:
*/2 * * * * DISPLAY=:0 XAUTHORITY=/home/andy/.Xauthority xdotool key Ctrl

Original post
In spite of not having any screen saver activate and in spite of setting "Blank Screen" to "Never' under Power options, my screen kept on blanking out after 1-3 minutes of watching TV using mythtv. I tried a lot of things, including using xset, but the only thing that worked in the end was editing my /etc/X11/xorg.conf (I use an nvidia driver) and adding the bits in red:
Section "ServerLayout"
    Identifier     " Configured"
    Screen      0  "Screen0" 0 0
    Screen      1  "Screen1" RightOf "Screen0"
    Screen      2  "Screen2" RightOf "Screen1"
    InputDevice    "Mouse0" "CorePointer"
    InputDevice    "Keyboard0" "CoreKeyboard"
    Option "BlankTime" "0"
    Option "StandbyTime" "0"
    Option "SuspendTime" "0"
    Option "OffTime" "0"
    Option "DPMS" "0"
Now the screen never blanks when watching TV using mythtv. On the other hand, it still blanks when watching DVDs in VLC...

18 January 2016

626. Briefly: Gaussian and cloud computing -- Gaussian G09D with Slurm on aws/ec2

Note: you may want to install awscli and euca2ools. I didn't, so I don't actually know whether they are useful.

My instructions are quite rudimentary since I don't have much time to write these blog posts anymore. Hopefully there's enough information to get you through.

Either way, sign up for AWS. If you already have an amazon ID I think you can use that. Go to

Select Launch an Instance and pick the ubuntu AIM and do Launch and Review. I launched it as a t2.micro instance type, as it is free and it's sufficient for set up but not to run jobs.

Hit launch, and create a new key pair. I called mine myfirstkeypair and saved the pem file in my ~/Downloads folder

In my Downloads folder:
ssh -i "myfirstkeypair.pem"
I then set a password in the ubuntu AWS image:
sudo passwd ubuntu

I added my to ~/.ssh/authorized_keys on the ubuntu AWS image to make logging in via ssh easier -- that way I won't need the pem file.

Set up Gaussian
I then connected with SCP and uploaded my gaussian files -- I went straight for EM64T G09D. It went quite fast at +5 MB/s

scp E6L-103X.tgz

Once that was done, on the ubuntu AWS instance I did:
sudo apt-get install csh 
sudo mkdir /opt/gaussian
cd /opt 
sudo chown ubuntu gaussian -R
cd /opt/gaussian
cp ~/E6L-103X.tgz .
tar xvf E6L-103X.tgz
cd g09
csh bsd/install

echo 'export GAUSS_EXEDIR=/opt/gaussian/g09/bsd:/opt/gaussian/g09/local:/opt/gaussian/g09/extras:/opt/gaussian/g09' >> ~/.bashrc
echo 'export GAUSS_SCRDIR=/home/ubuntu/scratch' >> ~/.bashrc
echo 'export PATH=$PATH:/opt/gaussian/g09' >> ~/.bashrc
source ~/.bashrc 
mkdir ~/scratch ~/jobs

NOTE that you can't run any gaussian jobs under a t2.micro instance. You will have to stop and relaunch as at least a t2.small instance or the jobs will be 'Killed' (that's what is echoed in the terminal when you try to run)
Note that if you terminate an image it will be deleted.

Stop the image and then create a snapshot or an image from it to keep everything you've installed.

Set up Slurm
You'll want a queue manager so that you can launch several jobs in serial. Also, you can set up your script so that it shuts down the image when your job is done to save money.

sudo apt-get update
sudo apt-get install slurm-llnl

ControlMachine=localhost ControlAddr= MpiDefault=none ProctrackType=proctrack/pgid ReturnToService=2 SlurmctldPidFile=/var/run/slurm-llnl/ SlurmdPidFile=/var/run/slurm-llnl/ SlurmdSpoolDir=/var/lib/slurm-llnl/slurmd SlurmUser=slurm StateSaveLocation=/var/lib/slurm-llnl/slurmctld SwitchType=switch/none TaskPlugin=task/none FastSchedule=1 SchedulerType=sched/backfill SelectType=select/linear AccountingStorageType=accounting_storage/none ClusterName=rupert JobAcctGatherType=jobacct_gather/none SlurmctldLogFile=/var/log/slurm-llnl/slurmctld.log SlurmdLogFile=/var/log/slurm-llnl/slurmd.log NodeName=localhost NodeAddr= PartitionName=All Nodes=localhost
sudo /usr/sbin/create-munge-key
Edit /etc/default/munge:
Then run
sudo service slurm-llnl restart
sudo service munge restart 
Test using slurm.batch
#!/bin/bash # #SBATCH -p All #SBATCH --job-name=test #SBATCH --output=res.txt # #SBATCH --ntasks=1 #SBATCH --time=10:00 srun hostname srun sleep 60
and submit with
sbatch slurm.batch
                 2       All     test   ubuntu  R       0:08      1 localhost

#!/bin/csh #SBATCH -p All #SBATCH --time=9999999 #SBATCH --output=slurm.out #SBATCH --job-name=benchmark setenv GAUSS_SCRDIR /home/ubuntu/scratch setenv GAUSS_EXEDIR /opt/gaussian/g09/bsd:/opt/gaussian/g09/local:/opt/gaussian/g09/extras:/opt/gaussian/g09 /opt/gaussian/g09/g09< > benchmark.out
Using the same opt/freq benchmark as in post 621.

c4.2xlarge 2h 11 min [1h 20 min] 8 vcpu/16 Gb
c4.4xlarge 1h 15 min [     44 min] 16 vcpu/32 Gb
c4.8xlarge      41 min [     25 min] 36 vcpu/60 Gb

It scales surprisingly well, although not perfectly linearly. It's clear that it's cheaper to use a smaller instance, so if time isn't critical or the larger memory isn't needed, c4.8xlarge is not the first choice.

You might want to use dropbox to transfer files back and forth, especially finished job files (useful if you shut down the machine using a slurm script as shown below)

cd ~ && wget -O - "" | tar xzf -
This computer isn't linked to any Dropbox account... Please visit to link this device. This computer isn't linked to any Dropbox account...

Open that link in a browser, then go back to the terminal.
wget -O - >
sudo mv /usr/local/bin
sudo chmod +x d/usr/local/bin/ autostart y

Now, since you don't want to use up space unnecessarily (you're paying for it after all), exclude as many directories as possible. To exclude all existing dropbox dirs, do
cd ~/Dropbox exclude add `ld -d */` exclude add `ld *.*` exclude list

Note that it can't handle directories with spaces in the name, so you'll need to polish the list by hand. Next create a directory where you want to run and store your jobs,e .g.
mkdir ~/Dropbox/aws_jobs

When you run a gaussian job, make sure to specify where the .chk files should end up, e.g.
so that you don't use up space/bandwidth for your chk files (unless of course you want to).

Stop after execution:
Use a batch script along these lines:
#!/bin/csh #SBATCH -p All #SBATCH --time=9999999 #SBATCH --output=slurm.out #SBATCH --job-name=benchmark setenv GAUSS_SCRDIR /home/ubuntu/scratch setenv GAUSS_EXEDIR /opt/gaussian/g09/bsd:/opt/gaussian/g09/local:/opt/gaussian/g09/extras:/opt/gaussian/g09 /opt/gaussian/g09/g09< > benchmark.out rm /home/ubuntu/scratch/*.* sudo shutdown -h now

17 August 2015

621. Very briefly: comparison of different hardware for a single G09 calculation

And another update:
* I've set up an Amazon EC2/AWS instance and have done some benchmarking there too: 
Exciting stuff.

Another update:
* I turned off hyperthreading on the i7-4930K to see whether it improved performance. The geometry optimisation took 2 min (4%) longer but the overall runtime was 2 min (2%) shorter. This is probably within the reproducibility error for the measurement.
* I've also added results for an i7-5820K without hyperthreading

Update: I spotted a few mistakes
* the L5430 job ran on a dual-socket machine, so I've multiplied the passmark by two and have replotted
* the X3480 job use the EM64T version of gaussian, not AMD64. I don't have a license to test that system using AMD64.

Original post:
There are lots of potential flaws when comparing the performance of a computational package on different hardware. Thus, it can be difficult to find examples online comparing different hardware using computational chemistry packages which makes it challenging to decide on what hardware to budget for.

So here's a simple comparison of a few different types of hardware for a geovib calculation in Gaussian.

All systems have spinning (7200 rpm) disks and use debian jessie (64). The systems haven't been optimised in any way.

All systems used G09D rev 01 AMD 64 unless otherwise indicated. The amount of time the geometry optimisation took is given within [].

2h 15 min. [1h 14 min.] Intel i7-4930K/ 32 Gb ram/ 12 threads
2h 23 min. [1h 25 min.] Intel i7-5820K/ 32 Gb ram/ 6 threads (HT turned off, NOTE)
3h 49 min. [2h 12 min.] AMD FX 8350/ 8 Gb ram/ 8 threads
4h 12 min. [2h 19 min.] Intel i5-2400/ 16 Gb ram/ 4 threads
4h 16 min. [2h 24 min.] AMD Phenom II X6 1055T/ 8 Gb ram/ 6 threads
4h 28 min. [2h 16 min.] dual-socket Intel Xeon L5430/ 16 Gb ram/ 8 threads-- rev A.02
4h 43 min. [2h 47 min.] AMD FX 8150/ 32 Gb ram/ 8 threads
9h 26 min. [5h 18 min.] AMD Athlon II X3 445/ 8 Gb ram/ 3 threads

I also tried the EM64T version of G09D rev 01 and got:
1h 43 min. [57 min.] i7-4930K/ 32 Gb ram/ 12 threads
3h 03 min. [1h 44 min.] i5-2400/16 Gb ram/ 4 threads
4h 21 min. [2h 27 min.] Xeon X3480/ 8 Gb ram/ 8 threads -- rev B.01

Just by switching from the AMD64 to the EM64T version we thus cut the calculation down to 75% of the time for the i7.

 I also turned off hyperthreading for the i7-4930K and ran with six threads using the EM64T version:
 1h 41 min. [59 min.] i7-4930K/ 32 Gb ram/ 6 threads
 1h 35 min. [58 min.] i7-5820K/ 32 Gb ram/ 6 threads (NOTE)

Here's a plot of the run times vs Passmark benchmarks (I've multiplied the Xeon L5430 passmark by 2):

Note that the Xeon L5430 and Xeon X340 store the job files on an NFS (scratch files are local) and are not using G09 rev D.01.

It'd be tempting to draw a line through the Athlon II to the I7-4930K, in which case the Phenom II X6 and the I5-2400 perform much, much better than they should based on the CPU Passmark alone.

Either way, these are my observations. No interpretations or opinion attached.

So what's  the benchmarking job that I used? I actually prefer not to reveal it, as it'd eventually point towards my identity (and you're not supposed to publish gaussian benchmarks...)

Suffice to say that it uses:
rpbe/def2-svp and 459 functions/759 primitives (46 atoms) with opt=(verytight) and integral(ultrafine)

620. Very Short: Setting up ORCA (comp. chem.) on debian jessie

Nothing odd here, but providing instructions is never a bad idea. Note that ORCA is binaries only i.e. no source code. It's one reason why I wouldn't want to rely on it as my main code for computational chemistry. However, I am interested in using it to compare select results from nwchem and g09, especially in cases where g09 and nwchem produce differing results.

Either way,. to download ORCA you need to join the ORCA forum:

Once you've joined you can go to Downloads and, well, download. I went for

sudo mkdir /opt/orca
sudo chown $USER /opt/orca
cp ~/Downloads/orca_3_0_3_linux_x86-64.tbz /opt/orca
cd /opt/orca
tar xvf orca_3_0_3_linux_x86-64.tbz
cd orca_3_0_3_linux_x86-64
ln -s orca orca_cc

The symmlink is there because debian already has a binary called orca.
And that's that installed. To add /opt/orca/orca_3_0_3_linux_x86-64 to PATH do

echo 'export PATH=$PATH:/opt/orca/orca_3_0_3_linux_x86-64' >> ~/.bashrc

To test, try one of the jobs on and run using
/opt/orca/orca_3_0_3_linux_x86-64/orca_cc test.inp
***************** * O R C A * ***************** --- An Ab Initio, DFT and Semiempirical electronic structure package --- ####################################################### # -***- # # Department of molecular theory and spectroscopy # # Directorship: Frank Neese # # Max Planck Institute for Chemical Energy Conversion # # D-45470 Muelheim/Ruhr # # Germany # # # # All rights reserved # # -***- # ####################################################### Program Version 3.0.3 - RELEASE - [..] ************************************************************ * Program running with 2 parallel MPI-processes * * working on a common directory * ************************************************************ ------------------------------------------------------------------------------ [..] ------------------------------------------------------------------------------ ORCA ELECTRIC PROPERTIES CALCULATION ------------------------------------------------------------------------------ Dipole Moment Calculation ... on Quadrupole Moment Calculation ... off Polarizability Calculation ... off GBWName ... test.gbw Electron density file ... test.scfp.tmp ------------- DIPOLE MOMENT ------------- X Y Z Electronic contribution: 0.00000 -0.00000 -0.00000 Nuclear contribution : -0.00000 0.00000 -0.00000 ----------------------------------------- Total Dipole Moment : -0.00000 0.00000 -0.00000 ----------------------------------------- Magnitude (a.u.) : 0.00000 Magnitude (Debye) : 0.00000 Timings for individual modules: Sum of individual times ... 45.891 sec (= 0.765 min) GTO integral calculation ... 5.511 sec (= 0.092 min) 12.0 % SCF iterations ... 26.269 sec (= 0.438 min) 57.2 % SCF Gradient evaluation ... 13.957 sec (= 0.233 min) 30.4 % Geometry relaxation ... 0.154 sec (= 0.003 min) 0.3 % ****ORCA TERMINATED NORMALLY**** TOTAL RUN TIME: 0 days 0 hours 0 minutes 50 seconds 359 msec

05 June 2015

611. Building ecce on debian jessie

* I've confirmed that ECCE built like this installs and works perfectly on a Thinkpad SL410 with intel graphics

* It also compiles and runs perfectly on a home built desktop with external nvidia card (GF119) using the binary non-free nvidia drivers

* It also compiles and runs perfectly on a home built desktop with onboard nvidia (GeForce 7025/nForce 630a) using the nouveau drivers. I had issues on this desktop before, but reinstalled debian jessie from scratch. Before that I used nvidia-legacy drivers, which may or may not  (probably not) have had something to do with it not working.

* UPDATE: instead of putting
#include <freetype.h>
the recommended method is to use
#include FT_FREETYPE_H
I've updated the below accordingly. Same goes for ftoutln.h vs FT_OUTLINE_H, but the latter didn't work (error saying #include must use "" or <>)

* UPDATE: If you're having issues with undetected -lGL and -lGLU in the wxwidgets step, it's because
667       endif
668       cat configure.orig | sed -e 's%^SEARCH_INCLUDE="\\%SEARCH_INCLUDE="$ECCE_HOME/${ECCE_SYSDIR}3rdparty/mesa/include \\%' >! configure
669       chmod a+x configure

needs to be changed to
667       endif
668       cat configure.orig | sed -e 's%^SEARCH_INCLUDE="\\%SEARCH_INCLUDE="$ECCE_HOME/${ECCE_SYSDIR}3rdparty/mesa/include $ECCE_HOME/${ECCE_SYSDIR}3rdparty/mesa/lib \\%' >! configure
669       chmod a+x configure
I had this issue on a debian wheezy system with the vendor nvidia libraries.
I wouldn't have spotted this bug otherwise.

* Another error from my debian wheezy nvidia system: if you get
checking how to run the C preprocessor... x86_64-linux-gnu-gcc -E ./configure: line 2880: syntax error near unexpected token `Using' ./configure: line 2880: `  AC_MSG_NOTICE(Using external PCRE library from $PCRE_CONFIG)'
then make sure you're not using autoconf2.13, which is an obsolete version. I think I have it due to my system originally being installed back in 2010.

* Finally, I'm currently  working on fixing minor things that have been nagging me in ecce. One is the basis set quicklist (in src/dsm/edsiimpl/ICalcUtils.C), but obviously that's a personal preference. A more serious one is the 256 character limit for csh commands:
Exceeds maximum C shell command length of 256 characters
Note that this isn't the C shell complaining -- this is a built-in limit in ecce (in src/comm/rcommand/RCommand.C). I've changed that limit to 16384 characters (the real limit is much, much higher)
I've also added two basis sets to ECCE, and have tinkered with the exchange-correlation functionals.

I'll try to push the fixes back upstreams when I'm ready if they'll accept them; otherwise I'll create my own github/sourceforge repo.

Building ECCE on debian wheezy was a breeze. Building ECCE on debian jessie was painful.

In the end it boiled down to two things:
*freetype headers are no longer in freetype2/freetype/ but in freetype2/

*-Wformat-security is turned on by default

mkdir ~/tmp/ecce_compile -p
cd ~/tmp/ecce_compile
sudo apt-get install bzip2 build-essential autoconf libtool ant pkg-config
sudo apt-get install gtk+-2.0-dev libxt-dev csh gfortran openjdk-7-jdk python-dev
sudo apt-get install libjpeg-dev imagemagick xterm libfreetype6-dev libfl-dev libtool-bin

As usual I'm not 100% sure when it comes to the necessary packages. libfl-dev might not be needed.

Download the ECCE source code and put the ecce-v7.0-src.tar.bz2 file in ~/tmp/ecce_compile. Put the file (see below in this post for the code) in ~/tmp/ecce_compile. Then do
tar xvf ecce-v7.0-src.tar.bz2 
cd ecce-v7.0/
export ECCE_HOME=`pwd`
cd build/

You'll now step through a list over programs and libraries that are needed and what ECCE can find. If you're having issues with e.g. javac and java being different versions, use sudo update-alternative --config javac.

At the end you'll be asked whether to skip these checks next time -- answer y(es).

Next do
./build_ecce|tee xerxes.log && ./build_ecce |tee mesa.log && ./build_ecce |tee wxwidgets.log
sh ../../ && ./build_ecce|tee wxpython.log 
./build_ecce|tee httpd.log && ./build_ecce|tee ecce.log

If all went well you've managed to build the ecce binaries. If not, go through wxpython.log and check for errors relating to format-security statements. Then go through ecce.log and look for issues with freetype.

What to do with the binaries? Follow one of the many ECCE installation posts on this blog, e.g.

NOTE that if you get ''Invalid null command." trying to execute install_ecce.v7.0.csh, install tcsh and do
tcsh install_ecce.v7.0.csh

The file -- when copying, make sure to check that the lines end where they are supposed to and not broken up.
#!/bin/bash cp ${ECCE_HOME}/build/3rdparty-dists/wxPython-src- ${ECCE_HOME}/3rdparty/build/ cd ${ECCE_HOME}/3rdparty/build/ tar xvf wxPython-src- rm wxPython-src- cd ${ECCE_HOME}/3rdparty/build/wxPython-src- grep -rsl "PyErr_Format(PyExc_RuntimeError, mesg)" *|xargs -I {} sed -i 's/PyErr_Format(PyExc_RuntimeError, mesg)/PyErr_Format(PyExc_RuntimeError, "%s", mesg)/g' {} cd ${ECCE_HOME}/3rdparty/build/wxPython-src- sed -i 's/wxLogFatalError(m)/wxLogFatalError("%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogError(m)/wxLogError("%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogWarning(m)/wxLogWarning("%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogMessage(m)/wxLogMessage("%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogInfo(m)/wxLogInfo("%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogDebug(m)/wxLogDebug("%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogVerbose(m)/wxLogVerbose("%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogStatus(pFrame, m)/wxLogStatus(pFrame, "%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogStatus(m)/wxLogStatus("%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogSysError(m)/wxLogSysError("%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogGeneric(level, m)/wxLogGeneric(level, "%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogTrace(mask, m)/wxLogTrace(mask, "%s", m.c_str())/g' _misc_wrap.cpp cd ${ECCE_HOME}/src grep -srl "<freetype/freetype.h>" |xargs -I {} sed -i 's,<freetype/freetype.h>,FT_FREETYPE_H,g' {} grep -srl "freetype/" |xargs -I {} sed -i 's,freetype/,,g' {} cd ${ECCE_HOME}/build

What I found during troubleshooting:

In files such as:
there are sections which look like this:
863 } else { 864 PyErr_Format(PyExc_RuntimeError, mesg); 865 }
Compiling with -Wformat-security means that you'll have to patch all those expression to
863 } else { 864 PyErr_Format(PyExc_RuntimeError, "%s", mesg); 865 }
There were similar issue with wxLog*(m) statements in other files, e.g.
3rdparty/build/wxPython-src- -> ("%s", m.c_str()) 3093 m.Replace(wxT("%"), wxT("%%")); 3094 wxLogFatalError(m); 3095 } .. 3177 m.Replace(wxT("%"), wxT("%%")); 3178 wxLogTrace(mask, m); 3179 }

04 June 2015

609. NBO6 on a debian cluster (/w g09)

Curse blogspot and the lack of revision control and backups! I lost my post when it was almost finished.

So here's a briefer version. I have bought NBO6 and I want to integrate it with gaussian G09 rev. D (you can't use it directly with earlier binary versions)

My 'instructions' are basically copy/pasted from the NBO6 installation instructions -- this is a tl;dr version.
sudo cp nbo6.0-bin-linux-x86_64.tar.gz /opt/ cd /opt/ sudo tar xvf nbo6.0-bin-linux-x86_64.tar.gz sudo chown $USER:$USER nbo6 -R vim nbo6/bin/gaunbo6
3 set INT = i8 4 set BINDIR = /opt/nbo6/bin
In your queue file add
set path = ( /opt/nbo6/bin $path )
In my case, as I use ECCE I edited my apps/siteconfig/CONFIG.node files:
Gaussian-03Command{ set path = ( /opt/nbo6/bin $path ) setenv GAUSS_SCRDIR /home/me/scratch setenv GAUSS_EXEDIR /opt/gaussian/g09d/g09/bsd:/opt/gaussian/g09d/g09/local:/opt/gaussian/g09d/g09/extras:/opt/gaussian/g09d/g09 /opt/gaussian/g09d/g09/g09< $infile > $outfile echo 0 }
I then tested it by running a basic gaussian calculation:
%Chk=H2O_631g.chk #P rOPBE/6-31G 6D 10F SCRF=(PCM,Solvent=water) Punch=(MO) pop=(nbo6) H2O 6-31G 0 1 ! charge and multiplicity O 0.00000 0.00000 0.118491 H 0.00000 0.754898 -0.473964 H 0.00000 -0.754898 -0.473964
and got
... 411 fchk file "/home/me/scratch/Gau-24659.EFC" 412 mat. el file "/home/me/scratch/Gau-24659.EUF" 413 414 Writing Wrt12E file "/home/me/scratch/Gau-24659.EUF" 415 Gaussian matrix elements Version 1 NLab= 7 Len12L=8 Len4L=8 416 Write GAUSSIAN SCALARS from file 501 offset 0 to matrix element file. .. 429 Write ALPHA FOCK MATRIX from file 10536 offset 0 to matrix element file. 430 No 2e integrals to process. 431 Perform NBO analysis... 432 433 *********************************** NBO 6.0 *********************************** 434 N A T U R A L A T O M I C O R B I T A L A N D 435 N A T U R A L B O N D O R B I T A L A N A L Y S I S 436 ********************* Me ********************* .. 447 Filename set to /home/me/scratch/Gau-24659 .. 620 ------------------------------- 621 Total Lewis 9.99612 ( 99.9612%) 622 Valence non-Lewis 0.00029 ( 0.0029%) 623 Rydberg non-Lewis 0.00358 ( 0.0358%) 624 ------------------------------- 625 Total unit 1 10.00000 (100.0000%) 626 Charge unit 1 0.00000 627 628 $CHOOSE 629 LONE 1 2 END 630 BOND S 1 2 S 1 3 END 631 $END 632 633 Maximum scratch memory used by NBO was 62605 words 634 Maximum scratch memory used by G09NBO was 9032 words ...

19 November 2014

602. Surviving bankid + rant. Exporting bankid files from linux to windows

This post probably isn't interesting unless you live/have lived in Sweden (Scandinavia)/deal with Swedish(Scandinavian) banks and their unbelievably crappy electronic ID solution.

This post will not show you how to use BankID on Linux -- it will show you how to move bankid files from debian linux to windows 7. Nor is the method elegant as it involves using two VMs.

I should make a long rant about how the company, Finansiell ID teknik, behind BankID should be sued into oblivion and the Swedish  politicians allowing it to be used to access government service without requiring open specifications be jailed indefinitely. But I'm tired.

[looks like I got into a rant after all]
rant begin:
Long story short: BankID has always been a real pain in the backside to use on linux, and now support has ended altogether. Support for Windows XP has been dropped as well, which will become relevant later.

No more linux announcement:

Either way, there's no linux-friendly solution out there, as in moving from v4 to v5 of bankid the format has changed completely. To my understanding, that means that fribid also will not work with providers requiring bankid versions newer than

There's a linux-friendly solution: mobile bankid, which runs on android. However, it seems to require a Swedish (or northern European) SIM card. Actually, I have no idea how it works since there's hardly any bloody information out there -- the website is incredibly bad. The most detailed info I've found is this, which says that it's SIM card and service provider dependent: On the other hand, this page says it isn't:

[NOPE -- that's not how it works -- it's actually a lot better. See here instead: linktocome ]

In the end it doesn't matter since you'll have to walk into a Swedish bank in person to order a new bankid set-up code. Not something you can do if you're abroad.

There's one other solution available -- ID via the Swedish tax office. Unfortunately it is only available for people residing in Sweden. Anyone who has business with Sweden but lives abroad is SOL.
rant end.

Step 1. Windows 7 in a VM -- overview
So, I had to find a pragmatic solution, and quickly since my current ID key is expiring. I have an old Windows XP disk that I could install in a VM, but since XP isn't supported anymore, that wouldn't work anyway.

I then decided to see how much a copy of Windows 7 costs. I'm not keen on throwing money at MS, but I was still at the point of simply investigating solutions. According to  the cheapest copy is $124. Not keen.

I am surrounded by computers with Windows 7 Pro OA stickers (that are running debian), so I figured if I could only get my hands on a DVD I could use the license that has already been paid for. I also had a vague memory of the Windows 7 isos being freely and legally available from Microsoft via download.

Using google a bit I eventually found it:
which provides Windows 7 isos freely and legally via Digital River. Note that you'll still need to find your own activation key.

Activation key -- you can either:
* get that legally by paying for it. Not interested in paying that much money for something I'm not going to use much.

* use google for a few minutes, find tons of activation keys which appear to work during installation but really don't allow you to activate your copy (go to Computer, Properties and click on Activate Now). This is obviously not legal.

* Put in an installation key (D4F6K-QK3RD-TMVMJ-BBMRX-3MBMV... read more) which allows you to finish the installation but won't activate your copy. Actually, reading the linked post you might not have to enter anything.

Either way, after 30 days your copy will expire. You can still use it -- the only things that change is the background (goes black), you can't install updates anymore, and Aero turns off. See here. This is fine by me.

As OA licenses are limited and I really don't care about using Windows for anything but bankid, I went with the last option.

Installing Windows 7 Ultimate 64 bit in VirtualBox went without a hitch, and I awarded it 30 Gb HDD and 2 gb RAM (I have 4 Tb HDD and 8 Gb RAM on my system). So far so good..

From within Windows 7 I then downloaded bankid v 6.1 from here: (ignore the whole linux/ubuntu link -- that's for v 4 which isn't supported by most banks anymore...)

Step 2. Ubuntu (yup) in a VM *yup).
At this point I figured I was doing pretty well. I connected a USB stick to my computer (i.e. debian) and ran persadm.
me@beryllium:/media/highio$ persadm export BankID Security Application 4.19.1 Available tokens: 0: (140110 yy.xx) Me - BankID on file 1: (130304 yy.xx) Me - BankID on file Choose token: 0 Enter pin: Enter removable media export directory (must exist): /media/fat32/bid Failed to export token. The reason might be that the export directory is not on a removable media, or that you don't have permission to write to it.
No matter what I tried in terms of permissions and destination file systems made any difference.

In desperation I then copied the ~/.personal structure containing the bankid keys from my debian box to my USB stick, fired up an old Ubuntu virtual machine with bankid installed (you may want to download bankid for ubuntu from here for THAT purpose:

Running persadmin in the Ubuntu 10.04 LTS 32 bit VM worked perfectly, and I now had the exported .nge file in a folder on my USB stick.
me@me-desktop:~$ persadm export BankID Security Application 4.19.1 Available tokens: 0: (140110 yy.xx) Me - BankID on file 1: (130304 yy.xx) Me - BankID on file Choose token: 0 Enter pin: Enter removable media export directory (must exist): /media/fat32/bid Successfully exported token.
In retrospect you might be able to do this in a chrooted ubuntu instance under debian. Who knows?

Step 3. Importing the BankID key files
I then fired up the Windows 7 VM, started BankID and went to File/Preferences. Under locations I added the folder on the USB stick that held the exported BankID file, clicked Add, then Save, and everything was good.

Well, apart from the fact that I had to use two VMs, and am now stuck with a Windows 7 VM.

My Win 7 .vdi file is about 9 Gb, which as a tar.gz file is compressed to 3.8 Gb. I've backed it up in three different locations, so hopefully there won't be any issues with losing the bankid due to MS update idiocies.

15 September 2014

594. Very, very briefly: Speedtest your WLAN

As shown here:

Install iperf: sudo apt-get install iperf

Open two terminal windows on the same computer:

Terminal 1$ iperf -s -u -i1
Server listening on UDP port 5001
Receiving 1470 byte datagrams
UDP buffer size:  208 KByte (default)
Terminal 2$ iperf -c -u -i1 -l 1300 -b 100m -t 600
Client connecting to, UDP port 5001
Sending 1300 byte datagrams
UDP buffer size:  208 KByte (default)
[  3] local port 56520 connected with port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 1.0 sec  4.46 MBytes  37.4 Mbits/sec
[  3]  1.0- 2.0 sec  4.45 MBytes  37.3 Mbits/sec
[  3]  2.0- 3.0 sec  4.28 MBytes  35.9 Mbits/sec
[  3]  3.0- 4.0 sec  4.32 MBytes  36.2 Mbits/sec
[  3]  4.0- 5.0 sec  4.42 MBytes  37.1 Mbits/sec
[  3]  5.0- 6.0 sec  4.35 MBytes  36.5 Mbits/sec
[  3]  6.0- 7.0 sec  4.39 MBytes  36.8 Mbits/sec
[  3]  7.0- 8.0 sec  4.35 MBytes  36.5 Mbits/sec
[  3]  8.0- 9.0 sec  4.41 MBytes  37.0 Mbits/sec

Where is my wireless router.

593. nwchem 6.5 on debian jessie and wheezy

A new version of nwchem is out now, and this time we're seeing a new solvation model: SMD! Given how long it took to get COSMO right, the involvement of Truhlar in implementing SMD is a very good thing.

Either way, in order to be able to compile nwchem 6.5 on wheezy you'll need to do a few things:
sudo apt-get install build-essential libopenmpi-dev openmpi-bin
wget -O Nwchem-6.5.revision26243-src.2014-09-10.tar.gz
tar xvf Nwchem-6.5.revision26243-src.2014-09-10.tar.gz 
cd Nwchem-6.5.revision26243-src.2014-09-10/

Create a patch file, e.g. 6.5.patch with the following content (most of the patches are for compatibility with gabedit, but a few of them are needed in order to compile nwchem on debian):
diff -rupN src.original/config/makefile.h src/config/makefile.h --- src.original/config/makefile.h 2014-09-15 12:28:58.153787833 +1000 +++ src/config/makefile.h 2014-09-15 12:30:58.745787161 +1000 @@ -2208,7 +2208,7 @@ endif ifeq ($(BUILDING_PYTHON),python) # EXTRA_LIBS += -ltk -ltcl -L/usr/X11R6/lib -lX11 -ldl - EXTRA_LIBS += -lnwcutil -lpthread -lutil -ldl -lz + EXTRA_LIBS += -lnwcutil -lpthread -lutil -ldl -lz -lssl LDOPTIONS = -Wl,--export-dynamic endif ifeq ($(NWCHEM_TARGET),CATAMOUNT) diff -rupN src.original/ddscf/movecs_pr_anal.F src/ddscf/movecs_pr_anal.F --- src.original/ddscf/movecs_pr_anal.F 2014-09-15 12:28:58.229787833 +1000 +++ src/ddscf/movecs_pr_anal.F 2014-09-15 12:32:33.801786632 +1000 @@ -195,7 +195,7 @@ c 22 format(1x,2(' Bfn. Coefficient Atom+Function ',5x)) write(LuOut,23) 23 format(1x,2(' ----- ------------ ---------------',5x)) - do klo = 0, min(n-1,9), 2 + do klo = 0, min(n-1,199), 2 khi = min(klo+1,n-1) write(LuOut,2) ( $ int_mb(k_list+k)+1, diff -rupN src.original/ddscf/rohf.F src/ddscf/rohf.F --- src.original/ddscf/rohf.F 2014-09-15 12:28:58.229787833 +1000 +++ src/ddscf/rohf.F 2014-09-15 12:33:12.897786414 +1000 @@ -153,7 +153,7 @@ c ilo = 1 ihi = nmo endif - call movecs_print_anal(basis, ilo, ihi, 0.15d0, g_movecs, + call movecs_print_anal(basis, ilo, ihi, 0.01d0, g_movecs, $ 'ROHF Final Molecular Orbital Analysis', $ .true., dbl_mb(k_eval), oadapt, int_mb(k_irs), $ .true., dbl_mb(k_occ)) diff -rupN src.original/ddscf/scf_vec_guess.F src/ddscf/scf_vec_guess.F --- src.original/ddscf/scf_vec_guess.F 2014-09-15 12:28:58.229787833 +1000 +++ src/ddscf/scf_vec_guess.F 2014-09-15 12:34:41.533785920 +1000 @@ -511,19 +511,19 @@ c nprint = min(nclosed+nopen+30,nmo) if (scftype.eq.'RHF' .or. scftype.eq.'ROHF') then call movecs_print_anal(basis, 1, - & nprint, 0.15d0, g_movecs, + & nprint, 0.01d0, g_movecs, & 'ROHF Initial Molecular Orbital Analysis', & .true., dbl_mb(k_eval), oadapt, int_mb(k_irs), & .true., dbl_mb(k_occ)) else nprint = min(nalpha+20,nmo) call movecs_print_anal(basis, max(1,nbeta-20), - & nprint, 0.15d0, g_movecs, + & nprint, 0.01d0, g_movecs, & 'UHF Initial Alpha Molecular Orbital Analysis', & .true., dbl_mb(k_eval), oadapt, int_mb(k_irs), & .true., dbl_mb(k_occ)) call movecs_print_anal(basis, max(1,nbeta-20), - & nprint, 0.15d0, g_movecs(2), + & nprint, 0.01d0, g_movecs(2), & 'UHF Initial Beta Molecular Orbital Analysis', & .true., dbl_mb(k_eval+nbf), oadapt, int_mb(k_irs+nmo), & .true., dbl_mb(k_occ+nbf)) diff -rupN src.original/ddscf/uhf.F src/ddscf/uhf.F --- src.original/ddscf/uhf.F 2014-09-15 12:28:58.229787833 +1000 +++ src/ddscf/uhf.F 2014-09-15 12:35:25.225785676 +1000 @@ -144,11 +144,11 @@ C enddo ihi = max(ihi-1,1) 9611 continue - call movecs_print_anal(basis, ilo, ihi, 0.15d0, g_movecs, + call movecs_print_anal(basis, ilo, ihi, 0.01d0, g_movecs, $ 'UHF Final Alpha Molecular Orbital Analysis', $ .true., dbl_mb(k_eval), oadapt, int_mb(k_irs), $ .true., dbl_mb(k_occ)) - call movecs_print_anal(basis, ilo, ihi, 0.15d0, g_movecs(2), + call movecs_print_anal(basis, ilo, ihi, 0.01d0, g_movecs(2), $ 'UHF Final Beta Molecular Orbital Analysis', $ .true., dbl_mb(k_eval+nbf), oadapt, int_mb(k_irs+nmo), $ .true., dbl_mb(k_occ+nbf)) diff -rupN src.original/mcscf/mcscf.F src/mcscf/mcscf.F --- src.original/mcscf/mcscf.F 2014-09-15 12:28:58.441787832 +1000 +++ src/mcscf/mcscf.F 2014-09-15 12:35:55.233785509 +1000 @@ -723,7 +723,7 @@ c if (util_print('final vectors analysis', print_default)) $ call movecs_print_anal(basis, $ max(1,nclosed-10), min(nbf,nclosed+nact+10), - $ 0.15d0, g_movecs, 'Analysis of MCSCF natural orbitals', + $ 0.01d0, g_movecs, 'Analysis of MCSCF natural orbitals', $ .true., dbl_mb(k_evals), .true., int_mb(k_sym), $ .true., dbl_mb(k_occ)) c diff -rupN src.original/nwdft/scf_dft/dft_mxspin_ovlp.F src/nwdft/scf_dft/dft_mxspin_ovlp.F --- src.original/nwdft/scf_dft/dft_mxspin_ovlp.F 2014-09-15 12:28:58.457787831 +1000 +++ src/nwdft/scf_dft/dft_mxspin_ovlp.F 2014-09-15 12:37:23.401785018 +1000 @@ -184,14 +184,14 @@ c call ga_sync() c call movecs_print_anal(basis,int_mb(k_non),int_mb(k_non) - & ,0.15d0,g_alpha,'Alpha Orbitals without Beta Partners', + & ,0.01d0,g_alpha,'Alpha Orbitals without Beta Partners', & .false., 0.0 ,.false., 0 , .false., 0 ) c if (nct.GE.2) then do i = 2,nct ind = int_mb(k_non+i-1) call movecs_print_anal(basis,ind,ind - & ,0.15d0,g_alpha,' ', + & ,0.01d0,g_alpha,' ', & .false., 0.0 ,.false., 0 , .false., 0 ) enddo endif @@ -350,7 +350,7 @@ c endif c endif c 9990 format(/,18x,'THERE ARE',i3,1x,'UN-PARTNERED ALPHA ORBITALS') c - call movecs_print_anal(basis, 1, nalp, 0.15d0, g_ualpha, + call movecs_print_anal(basis, 1, nalp, 0.01d0, g_ualpha, & 'Alpha Orb. w/o Beta Partners (after maxim. alpha/beta overlap)', & .false., 0.0 ,.false., 0 , .false., 0 ) c diff -rupN src.original/nwdft/scf_dft/dft_scf.F src/nwdft/scf_dft/dft_scf.F --- src.original/nwdft/scf_dft/dft_scf.F 2014-09-15 12:28:58.457787831 +1000 +++ src/nwdft/scf_dft/dft_scf.F 2014-09-15 12:38:29.713784648 +1000 @@ -2097,7 +2097,7 @@ c endif c ... jochen: replaced 0.15 at the end of the line with 'tanalyze' c which can be read from input -c call movecs_print_anal(ao_bas_han, ilo, ihi, 0.15d0, +c call movecs_print_anal(ao_bas_han, ilo, ihi, 0.01d0, call movecs_print_anal(ao_bas_han, ilo, ihi, tanalyze, & g_movecs(ispin), & blob, diff -rupN src.original/nwdft/scf_dft_cg/dft_cg_solve.F src/nwdft/scf_dft_cg/dft_cg_solve.F --- src.original/nwdft/scf_dft_cg/dft_cg_solve.F 2014-09-15 12:28:58.445787832 +1000 +++ src/nwdft/scf_dft_cg/dft_cg_solve.F 2014-09-15 12:39:25.497784337 +1000 @@ -193,7 +193,7 @@ c blob = 'DFT Final Beta Molecular Orbital Analysis' endif call movecs_fix_phase(g_movecs(ispin)) - call movecs_print_anal(basis, ilo, ihi, 0.15d0, + call movecs_print_anal(basis, ilo, ihi, 0.01d0, & g_movecs(ispin),blob, & .true., dbl_mb(k_eval+(ispin-1)*nbf), & oadapt, int_mb(k_irs+(ispin-1)*nbf), @@ -216,7 +216,7 @@ c endif call movecs_fix_phase(g_movecs) blob = 'DFT Final Molecular Orbital Analysis' - call movecs_print_anal(basis, ilo, ihi, 0.15d0, + call movecs_print_anal(basis, ilo, ihi, 0.01d0, & g_movecs,blob, & .true., dbl_mb(k_eval), & oadapt, int_mb(k_irs),

Apply it with
patch -p0 < 6.5.patch

Then build using something along the lines of
export NWCHEM_TOP=`pwd`
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_MODULES="all python"
export PYTHONHOME=/usr
export BLASOPT="-L/opt/openblas/lib -lopenblas"

export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_LOC=/usr/lib/openmpi/lib
export MPI_INCLUDE=/usr/lib/openmpi/include
export LIBRARY_PATH="$LIBRARY_PATH:/usr/lib/openmpi/lib:/opt/openblas/lib"

export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"

cd $NWCHEM_TOP/src

make clean
make nwchem_config
make FC=gfortran 1> make.log 2>make.err

cd $NWCHEM_TOP/contrib
export FC=gfortran

See other posts for using different math libs: openblas, intel mkl, AMD acml

If you're building for Jessie, set the following flags before building:
export PYTHONCONFIGDIR=config-x86_64-linux-gnu

28 July 2014

586.Very Briefly: Gnuradio, RTL-SDR, GQRX and an R280T device on debian jessie

* I did this on debian jessie which at the time has gnuradio v 3.7.3-9+b1 and rtl-sdr v 0.5.3-3

* I'm having a lot of trouble getting gqrx working on debian wheezy even with backports. Whereas the backports versions of gnuradio, rtl-sdr and gqrx-sdr install just fine, when running gqrx I get the following error:
gqrx: symbol lookup error: /usr/lib/x86_64-linux-gnu/ undefined symbol: _ZN16QIODevicePrivate13putCharHelperEc
* gqrx won't work on debian wheezy systems with 2 Gb of memory for some reason. I get the same error as is shown in this post:!topic/gqrx/20F8RMWkNbU

The post:
I recently bought a DVB USB dongle based on RTL2832 and R280T: AU$11for a TV card which actually works flawlessly under linux really isn't bad!

As I already have a mythtv setup using leadtek DTV 1000S I was more interested in exploring the R820T dongle as a software defined radio (SDR).

 So, after glancing at I did the following

sudo apt-get install rtl-sdr gnuradio gnuradio-dev libgnuradio-osmosdr0.1.1.4 git libboost-dev liblog4cpp5-dev libboost-system-dev libboost-program-options-dev checkinstall
mkdir ~/tmp
cd ~/tmp
git clone gqrx.git
cd gqrx.git/
mkdir build
cd build
qmake ../
sudo checkinstall --install=no
0 - Maintainer: [ root@niobium ] 1 - Summary: [ gqrx ] 2 - Name: [ gqrx ] 3 - Version: [ 20140726 ] 4 - Release: [ 1 ] 5 - License: [ GPL ] 6 - Group: [ checkinstall ] 7 - Architecture: [ amd64 ] 8 - Source location: [ build ] 9 - Alternate source location: [ ] 10 - Requires: [ ] 11 - Provides: [ build ] 12 - Conflicts: [ ] 13 - Replaces: [ ]
sudo dpkg -i gqrx_20140726-1_amd64.deb

Starting gqrx and using it is easy:
So far I haven't managed to get anything other than regular commercial radio signals (I've only explored the FM band).

29 May 2014

579. TEMPerHum (0c45:7402 ) on debian linux wheezy

Yet another little usb device from PC sensors (I seem to have been giving them a fair amount of money recently ).

This is the device in question: (note that I buy my devices via ebay where the prices are apparently always the campaign ones -- I paid AUD25 , not USD80)

It didn't come in a box so I have no scans or shots to show.

[See here for the regular TEMPer USB device (0c45:7401), and here for the TEMPer 1K4  (0c45:7403) USB thermocouple reader. Note that since the temper-usb code is in  a lot of flux you can't use the line numbers in those posts directly -- you'll have to read and understand the code before pasting it in. Luckily, it's quite simple -- even I managed to sort it out!]

I couldn't get temper-usb to work even when making (what I consider) the necessary edits, but instead got lots of errors (including " usb.USBError: could not detach kernel driver from interface 1: No data available"). So I finally gave up.

Instead, web searching led me to -- one of the replies by eg1l spelled out the solution -- I'll grant myself the liberty to repost it here, but please remember where it originally came from and link to the original article (exclusively or in addition).

mkdir ~/tmp
cd ~/tmp/
sudo apt-get install libudev-dev libusb-1.0.0-dev libfox-1.6-dev autoconf cmake

git clone git://
cd hidapi/
cd libusb/
sudo make install

cd ../../

git clone git://
cd TEMPered/
mkdir build
cd build/
cmake ..
cd utils/
sudo ./tempered 
cd ../
sudo make install

sudo ln -s /usr/local/lib/x86_64-linux-gnu/ /usr/local/lib/
sudo ln -s /usr/local/lib/x86_64-linux-gnu/ /usr/local/lib/
sudo ldconfig

sudo tempered
0006:000e:01 0: temperature 21.75 °C, relative humidity 49.1%, dew point 10.6 °C

Create an 80-temper.rules file in /etc/udev/rules.d:
SUBSYSTEMS=="usb", ATTRS{idVendor}=="0c45", ATTRS{idProduct}=="7401", GROUP="users", MODE="0666"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="0c45", ATTRS{idProduct}=="7402", GROUP="users", MODE="0666"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="0c45", ATTRS{idProduct}=="7403", GROUP="users", MODE="0666"

Then do
sudo usermod -a -G users $USER

sudo service udev restart

Unplug and re-plug your device, then open a new terminal and you're set (type group to make sure that users show up).

me@boron:~$ tempered
0006:000f:01 0: temperature 23.34 °C, relative humidity 46.8%, dew point 11.3 °C

20 April 2014

573. PCSensors K-type USB thermocouple adapter TEMPer1K4 (0c45:7403) on debian

UPDATE 1 May 2014:
I've rewritten the code now to properly deal with the presence of both a 0c45:7401 and a 0c45:7403 simultaneously. Note that I'm not convince about the accuracy of the temperature readings (accuracy as in whether there's any systematic bias to the output. The reproducibility seems to be excellent though).

Original post:
I just received this usb thermocouple reader (TEMPer1k4):

About the product:
lsusb shows 0c45:7403 (Microdia Foot Switch)

dmesg shows
[13448.536120] usb 6-1: new low-speed USB device number 3 using uhci_hcd [13448.709126] usb 6-1: New USB device found, idVendor=0c45, idProduct=7403 [13448.709139] usb 6-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0 [13448.709146] usb 6-1: Product: TH1000isoV1.5 [13448.709152] usb 6-1: Manufacturer: RDing

Getting it to work:
You can use the same program as in this post: with some minor modifications. Before running
sudo python install

make some changes in temperusb/

Firstly, you need to allow for the detection of devices with the 0c45:7403 ID
15 VIDPIDs = [(0x0c45L,0x7401L),(0x0c45L,0x7403L)]
Secondly, you need to make sure that the program knows which device is which and treats them differently. Finally, you need to collect the right data-- the TEMPer1k4 returns a data_s with a length of 8 (caveat -- I haven't checked what the output from 0c45:7401 looks like. Maybe it too yields 8 fields), and the [2:4] data block contains the internal temperature of the USB device, while the [4:6] data block holds the thermocouple junction temperature. Of a sort -- you get a number which you need to translate into a temperature. See the bottom of this post for how I worked it all out. Please note that you should calibrate the thermocouple -- the uncorrected output seems to be at least 2-3 degrees too high.
 15 VIDPIDs = [(0x0c45L,0x7401L),(0x0c45L,0x7403L)]

 63     def getid(self):
 64         return self._device.idProduct

125             if self._device.idProduct==29697:
126                 temp_c = 125.0/32000.0*(struct.unpack('>h', data_s[2:4])[0])+0.0025
127                 if format == 'celsius':
128                     temp_c = temp_c * self._scale + self._offset
129                     return temp_c
130                 elif format == 'fahrenheit':
131                     return temp_c*1.8+32.0
132                 elif format == 'millicelsius':
133                     return int(temp_c*1000)
134                 else:
135                     raise ValueError("Unknown format")
136             elif self._device.idProduct==29699:
137                 temp_c = (struct.unpack('>h', data_s[4:6])[0])*0.25-2.00
138                 temp_internal=(125.0/32000.0*(struct.unpack('>h', data_s[2:4])[0]))+0.0025
139                 if format == 'celsius':
140                     temp_c = temp_c * self._scale + self._offset
141                     return temp_c
142                 elif format == 'fahrenheit':
143                     if self._device.idProduct==29697: #0x7401
144                         return temp_c*1.8+32.0
145                     elif self._device.idProduct==29699: #0x7403
146                         return temp_internal
 31     for i, dev in enumerate(devs):
 32         readings.append({'device': i,
 33                          'id':dev.getid(),
 34                          'temperature_c': dev.get_temperature(),
 35                          'temperature_f':
 36                          dev.get_temperature(format="fahrenheit"),
 37                          'ports': dev.get_ports(),
 38                          'bus': dev.get_bus()
 39                          })
 41     for reading in readings:
 42         if disp_ports:
 43             portinfo = " (bus %s - port %s)" % (reading['bus'],
 44                                                 reading['ports'])
 45         else:
 46             portinfo = ""
 47         if reading['id']==29697:
 48             print('Device #%i%s: %0.1f°C %0.1f°F'
 49                   % (reading['device'],
 50                      portinfo,
 51                      reading['temperature_c'],
 52                      reading['temperature_f']))
 53         elif reading['id']==29699:
 54             print('Device #%i%s: %0.1f°C %0.1f°C'
 55                   % (reading['device'],
 56                      portinfo,
 57                      reading['temperature_c'],
 58                      reading['temperature_f']))

Beyond this, follow the instructions in and make sure to set up a rules file with the correct USB ID for this device.


If you're curious about the details, have a look below.

Analysing USB traffic:
Since I wasn't quite sure where to start I figured the easiest approach would be to sniff the traffic between the usb device and the offically supported programme from PC Sensors. So I installed it Win XP in virtualbox, read a couple of temperatures and compared them with the captured data.

To capture data I did:
sudo apt-get install tshark
sudo modprobe usbmon
Bus 008 Device 003: ID 17ef:4815 Lenovo Integrated Webcam [R5U877] Bus 008 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 004 Device 043: ID 0c45:7403 Microdia Foot Switch Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
sudo tshark -D
1. eth0 2. wlan0 3. nflog 4. nfqueue 5. usbmon1 6. usbmon2 7. usbmon3 8. usbmon4 9. usbmon5 10. usbmon6 11. usbmon7 12. usbmon8 13. any 14. lo (Loopback)
touch 1.pcap chmod ugo+w 1.pcap sudo tshark -i usbmon4 -w $HOME/1.pcap

I opened the 1.pcap file in wireshark and looked for fields called leftover data.

Here are two examples of such 'leftover data' fields:
Host to USB: 01 80 00 00 00 00 00 00
USB to Host: 80 06 17 f0 00 5c 0f ff

Subjecting the system to different temperatures helped me figure out what fields where changing -- I've marked them in bold above (blue for the thermocouple, red for the internal temperature). Interestingly, I got overflow when freezing the thermocouple between two ice cubes, which indicated that there was a fairly high lower limit (1.5 degrees)
The following are some of the data I harvested. The temperature is in the first column, the hex code is in the second one and the decimal value is in the third one:
23.63 17a 378
23.69 17b 379
23.94 17f 383
23.75 17c 380
21.88 15e 350
22.31 165 357
22.88 16e 366

Thermocouple: 23.75 05f 95 23.50 05e 9 38.25 099 153 24.00 066 102 87.50 15e 350 79.50 13e 318
In other words, T(thermo)=output(thermo)*0.25+1.50 and T(int.)=(output(int.)-0.0025)/0.0625. Because the internal data is in fields 2:4 it's seen as e.g. 17b0 instead of 17b, so the decimal version needs to be divided by 16 (line 126 above).

I also set up an /etc/temper.conf file and did chown $USER /etc/temper.conf in order to be able to read it (must be a better way). I tried to calibrate the thermocouple in my kitchen with iced water and boiling water. The boiling water gave a reading of 102 degrees, while the iced water gave me 7.5 degrees Celsius even after what should've been long enough to achieve equilibrium. I'll calibrate it in the lab later. So for now a simple offset might be enough to give a working temperature.
temper-poll -p
Device #0 (bus 2 - port 2)
so temper.conf became
2-2: scale = 1.00, offset = -4.0

The temper-usb code is changing too quickly!. You'll thus need to read and understand the code rather than just pasting it in. Here are the full, edited an files.
# encoding: utf-8
from __future__ import print_function
from temper import TemperHandler

def main():
    th = TemperHandler()
    devs = th.get_devices()
    readings = []
    print("Found %i devices" % len(devs))

    for i, dev in enumerate(devs):
        readings.append({'device': i,
                         'temperature_c': dev.get_temperature(),

    for reading in readings:
  if reading['id']==29697:
   print('Device #%i: %0.1f°C %0.1f°F' % (reading['device'],
  elif reading['id']==29699:
   print('Device #%i: %0.1f°C %0.1f°C' % (reading['device'],
# encoding: utf-8
# Handles devices reporting themselves as USB VID/PID 0C45:7401 (mine also says RDing TEMPerV1.2).
# Copyright 2012, 2013 Philipp Adelt 
# This code is licensed under the GNU public license (GPL). See for details.

import usb
import sys
import struct

VIDPIDs = [(0x0c45L,0x7401L),(0x0c45L,0x7402L),(0x0c45L,0x7403L)]
TIMEOUT = 2000

class TemperDevice():
    def __init__(self, device):
        self._device = device
        self._handle = None

    def get_temperature(self, format='celsius'):
            if not self._handle:
                self._handle =
                except usb.USBError:
                except usb.USBError:
                self._handle.controlMsg(requestType=0x21, request=0x09, value=0x0201, index=0x00, buffer="\x01\x01", timeout=TIMEOUT) # ini_control_transfer

            self._control_transfer(self._handle, "\x01\x80\x33\x01\x00\x00\x00\x00") # uTemperatura
            self._control_transfer(self._handle, "\x01\x82\x77\x01\x00\x00\x00\x00") # uIni1
            self._control_transfer(self._handle, "\x01\x86\xff\x01\x00\x00\x00\x00") # uIni2
            self._control_transfer(self._handle, "\x01\x80\x33\x01\x00\x00\x00\x00") # uTemperatura
            data = self._interrupt_read(self._handle)
            data_s = "".join([chr(byte) for byte in data])

            if self._device.idProduct==29697: 
    temp_c = 125.0/32000.0*(struct.unpack('>h', data_s[2:4])[0])+0.0025
    if format == 'celsius':
     return temp_c
    elif format == 'fahrenheit':
     return temp_c*1.8+32.0
    elif format == 'millicelsius':
     return int(temp_c*1000)
     raise ValueError("Unknown format")
            elif self._device.idProduct==29699: 
    temp_c = (struct.unpack('>h', data_s[4:6])[0])*0.25-4.00
    temp_internal=(125.0/32000.0*(struct.unpack('>h', data_s[2:4])[0]))+0.0025
    if format == 'celsius':
     return temp_c
    elif format == 'fahrenheit':
     if self._device.idProduct==29697: #0x7401
      return temp_c*1.8+32.0
     elif self._device.idProduct==29699: #0x7403
      return temp_internal
    elif format == 'millicelsius':
     return int(temp_c*1000)
     raise ValueError("Unknown format")
        except usb.USBError, e:
            if "not permitted" in str(e):
                raise Exception("Permission problem accessing USB. Maybe I need to run as root?")
    def getid(self):
        #print self._device.idProduct
        return self._device.idProduct

    def close(self):
        if self._handle:
            except ValueError:
            self._handle = None

    def _control_transfer(self, handle, data):
        handle.controlMsg(requestType=0x21, request=0x09, value=0x0200, index=0x01, buffer=data, timeout=TIMEOUT)

    def _interrupt_read(self, handle):
        return handle.interruptRead(0x82, REQ_INT_LEN)

class TemperHandler():
    def __init__(self):
        busses = usb.busses()
        self._devices = []
        for bus in busses:
            self._devices.extend([TemperDevice(x) for x in bus.devices if (x.idVendor,x.idProduct) in VIDPIDs])

    def get_devices(self):
        return self._devices

04 January 2014

541. Setting up mythtv on debian with leadtek 1000s (SAA7130)

I've been using me-tv (and occasionally kaffeine and vlc) to watch DVB-T on linux for the better part of two years. While it's working absolutely fine, and I really like me-tv, I'd like to get it to work in mythtv as well in the hope that it will support closed captioning.

Note that mythtv is only available via the deb-multimedia repo and not in the offical debian repos. I first tried to install mythtv that way, and while I got pretty far (scanned for channels etc.) I couldn't get a picture when trying to watch TV using mythtv-frontend. The deb-multimedia packages also screwed up my graphics card set-up somehow, which didn't improve my mood. So while that's probably a viable method, I finally decided to compile things myself. Note that if you do keep deb-multimedia enable and do a dist-upgrade the mythplugins package below will be replaced, and with that the main mythtv package as well, leading to all kinds of fun. You might want to give mythplugins a higher version number than I've done below to circumvent that.


sudo dpkg-reconfigure dash

Select 'No', so that /bin/sh/ points to bash instead of dash.

The next port of call was 

sudo apt-get install build-essential liblircclient-dev libasound2-dev libdts-dev libdvdnav-dev \
 libxv-dev libxxf86vm-dev transcode libmp3lame-dev subversion qt4-dev-tools libqt4-dev libsamplerate0 \
 libxvidcore4 liba52-0.7.4-dev libfame-dev libcdio-dev msttcorefonts libasound2-doc libmad0-dev \
 libid3tag0-dev libvorbis-dev libflac-dev libcdaudio-dev libcdparanoia0-dev fftw3-dev libfaad-dev \
 libsmpeg-dev libmp4v2-dev libtag1-dev mysql-server libvisual-0.4-dev libexif-dev libxvmc-dev \
 libxinerama-dev uuid-dev libicu-dev 
sudo apt-get install yasm checkinstall gdb python-mysqldb python-urlgrabber libnet-upnp-perl
mkdir ~/tmp/mythtv -p
cd ~/tmp/mythtv
git clone -b fixes/0.27 git://
cd mythtv/mythtv/
./configure --enable-proc-opt
sudo checkinstall --fstrans=no --install=no
0 - Maintainer: [ andy@helium ] 1 - Summary: [ mythtv 0.27 ] 2 - Name: [ mythtv ] 3 - Version: [ 0.27 ] 4 - Release: [ 1 ] 5 - License: [ GPL ] 6 - Group: [ checkinstall ] 7 - Architecture: [ amd64 ] 8 - Source location: [ mythtv ] 9 - Alternate source location: [ ] 10 - Requires: [ ] 11 - Provides: [ mythtv ] 12 - Conflicts: [ ] 13 - Replaces: [ ]
Exclude files that are in the home directory.

sudo dpkg -i mythtv_0.27-1_amd64.deb
cd ../mythplugins/
sudo apt-get install libmysql++-dev python-oauth libdate-manip-perl libxml-simple-perl libimage-size-perl libdatetime-format-iso8601-perl libsoap-lite-perl libjson-perl 
sudo checkinstall --fsmetrans=no --install=no
0 - Maintainer: [ root@helium ] 1 - Summary: [ mythplugins 0.27 ] 2 - Name: [ mythplugins ] 3 - Version: [ 0.27 ] 4 - Release: [ 1 ] 5 - License: [ GPL ] 6 - Group: [ checkinstall ] 7 - Architecture: [ amd64 ] 8 - Source location: [ mythplugins ] 9 - Alternate source location: [ ] 10 - Requires: [ ] 11 - Provides: [ mythplugins ] 12 - Conflicts: [ ] 13 - Replaces: [ ]
sudo dpkg -i mythplugins_0.27-1_amd64.deb

Note that you may want to use a much higher version number for mythplugins than I've done above -- with 0.27 the deb-multimedia version of the package will take precendence, and cause all kinds of fun...

Setting up

sudo useradd mythtv
sudo usermod -a -G mythtv $USER
mkdir ~/.mythtv

Create a file called ~/.mythtv/mysql.txt:
where helium is my hostname.
Before you can run mythtv-setup you should prepare the mysql database.

mysql -u root -p
> show databases;
> create database mythconverg;
> create user 'mythtv'@'%' identified by 'mythtv';
> create user 'mythtv'@'localhost' identified by 'mythtv';
> set password for 'mythtv'@'%' = password('mythtv');
> set password for 'mythtv'@'localhost' = password('mythtv');
> connect mythconverg;
> grant all privileges on *.* to 'mythtv'@'%' with grant option;
> grant all privileges on *.* to 'mythtv'@'localhost' with grant option;
> flush privileges;
> exit;

Now run setup:

The first time I did this it asked about country and language, and then exited. I then ran it again and went through the setup as shown in the figures below. Note that I had to use the IP,, of my computer rather than (i.e. localhost).

I then ran

and let it run in a terminal, followed by

in another terminal. And it worked! I mean, not the first time, but by following the instructions above I actually got it to work -- up to a point. The audio wasn't working.

While TV is a visual medium it's still nice to have sound. So, in mythfrontend I went to Settings, Audio and picked Pulseaudio, since that's what I use by default anyway.

 I did have the occasional issue with sudden white noise, so I changed to ALSA, and all has been perfect ever since:

Test driving:
Everything seems to be working fine. Hit M for menu:
 You can choose subtitles (or just hit T):

To bring up the Electronic Program Guide (EPG) hit S:
See for a list of shortcuts.

Anyway, I'm still exploring. The one thing that's not working yet is the remote control, but that will be solved in due course.

Missing channels:
Mythtv didn't pick up channels Nine, Gem and Go. While scan found them without issue by doing 'scan /usr/share/dvb/dvb-t/au-Melbourne > channels.conf', I couldn't find an easy way to import the scan data.


Trying to import an existing scan in mythtv didn't allow me to provide a URL, but only showed two old scans pre-populating the import dialogue, which was both weird and annoying.

Instead the key was to add a new transport: fire up mythtv-setup, and go to the Channel editor. Note that I'm using a different skin/menu to the default one (Terra), but the process is the same.

See scan output above for the values to add here

 And you're pretty much done.

25 December 2013

538. Briefly: Sort folders before files in nautilus 3.8

I'm running debian jessie (current testing) on my laptop and after having held off upgrading for a while since I had to take it to a conference and didn't want to risk ending up with a broken system, I finally took the leap. I notice that there are a lot of references to systemd in dmesg, but haven't had a look at what it actually means -- are we past init and fully switched to systemd now? Or how do I go about modifying my network configuration if I can't use /etc/network/interfaces?

Anyway, one annoying little thing is that in Nautilus the folder content by default is arranged in alphabetical order, regardless of whether it's a file or a directory. The old behaviour was to arrange folders in alphabetical order, then files.

Here's how to get it back to 'normal' behaviour:
The new behaviour
Click on the 'Files' menu on the top desktop bar, select preferences:
Check 'Sort folders before files' to get back the normal behaviour
Check sort folders before files to make Nautilus behave well again

12 November 2013

529. Briefly: Error Writing spool: NT_STATUS_DISK_FULL

I recently had trouble printing on a networked printer at work, where we use 'Papercut' to share printers -- basically you submit your job, give your credentials, then run over to a printer and release the job.

Anyway, I suddenly had issues printing:

The solution:
I'm not entirely sure what fixed it, but here's what I did

lpstat showed a number of jobs that had been submitted to the printer, but couldn't be released:
me@beryllium:~/Downloads$ lpstat
global-mfp-1166         me              591872   Mon 11 Nov 2013 12:36:29 EST
global-mfp-1167         me              993280   Mon 11 Nov 2013 12:36:44 EST
global-mfp-1168         me             2014208   Mon 11 Nov 2013 12:36:59 EST
global-mfp-1169         me              871424   Mon 11 Nov 2013 12:37:17 EST
global-mfp-1170         me              573440   Mon 11 Nov 2013 12:37:31 EST
global-mfp-1171         me             1199104   Mon 11 Nov 2013 12:37:51 EST
global-mfp-1172         me              183296   Mon 11 Nov 2013 12:38:02 EST
global-mfp-1173         me              491520   Mon 11 Nov 2013 12:38:19 EST
global-mfp-1174         me             2035712   Mon 11 Nov 2013 12:38:38 EST
global-mfp-1175         me             2035712   Mon 11 Nov 2013 12:39:54 EST
global-mfp-1176         me              635904   Mon 11 Nov 2013 12:41:54 EST
global-mfp-1177         me              148480   Mon 11 Nov 2013 16:29:58 EST
I preceded to cancel all the jobs:
me@beryllium:~/Downloads$ cancel global-mfp-1166
me@beryllium:~/Downloads$ lpstat
global-mfp-1167         me              993280   Mon 11 Nov 2013 12:36:44 EST
global-mfp-1168         me             2014208   Mon 11 Nov 2013 12:36:59 EST
global-mfp-1169         me              871424   Mon 11 Nov 2013 12:37:17 EST
global-mfp-1170         me              573440   Mon 11 Nov 2013 12:37:31 EST
global-mfp-1171         me             1199104   Mon 11 Nov 2013 12:37:51 EST
global-mfp-1172         me              183296   Mon 11 Nov 2013 12:38:02 EST
global-mfp-1173         me              491520   Mon 11 Nov 2013 12:38:19 EST
global-mfp-1174         me             2035712   Mon 11 Nov 2013 12:38:38 EST
global-mfp-1175         me             2035712   Mon 11 Nov 2013 12:39:54 EST
global-mfp-1176         me              635904   Mon 11 Nov 2013 12:41:54 EST
global-mfp-1177         me              148480   Mon 11 Nov 2013 16:29:58 EST
me@beryllium:~/Downloads$ cancel global-mfp-1167
me@beryllium:~/Downloads$ cancel global-mfp-1168
me@beryllium:~/Downloads$ cancel global-mfp-1169
me@beryllium:~/Downloads$ cancel global-mfp-1170
me@beryllium:~/Downloads$ cancel global-mfp-1171
me@beryllium:~/Downloads$ cancel global-mfp-1172
me@beryllium:~/Downloads$ cancel global-mfp-1173
me@beryllium:~/Downloads$ cancel global-mfp-1174
me@beryllium:~/Downloads$ cancel global-mfp-1175
me@beryllium:~/Downloads$ cancel global-mfp-1176
me@beryllium:~/Downloads$ cancel global-mfp-1177
me@beryllium:~/Downloads$ lpstat

That didn't remove the error message, however. Opening Printers in Gnome(3) showed that the printer with an issues was set to 'off'. Unlocking and changing it to 'on' resolved the issue.