Wednesday, October 20, 2010

LaTeX and git


Some of you may remember our discussion at dinner about using git/SVN with LaTeX documents. One of the problems that was mentioned was line-based diff - whenever the formatting of the paragraph changes all of the lines are marked as modified, even if the content did not change.

* git diff --color-words

I found very nice solution to the problem: in git diff you can use the option --color-words which performs word-based diff and highlights the changes in colors.

* gitattributes

You can also put the following line into your .gitattributes file:

*.tex diff=tex

which treats some tags (likes \section{}) as separate words even if there is no whitespace in between.

I am almost converted to git - thanks for opening my eyes ;)



Wednesday, October 6, 2010

PyAS - nuggets

PyAS - Nuggets

Ipython cpaste magic func to paste in code.

cython -a -> annotated html with colours for how long things take ... click to expand

Tuesday, October 5, 2010

Git & github getting started

Global setup:

Download and install Git
git config --global "Your Name"
git config --global

Next steps:

mkdir test
cd test
git init
touch README
git add README
git commit -m 'first commit'
git remote add origin
git push origin master

Existing Git Repo?

cd existing_git_repo
git remote add origin
git push origin master

Importing a Subversion Repo?

Click here

When you're done:


Friday, October 1, 2010

Emacs tricks: cua-mode


You cut some code from a website and it has a few columns of junk at the front, enter cua-mode

M-x cua-mode

Set marker, C-RET enters column mode. Select the block, C-x to cut.

Saturday, September 11, 2010

python crypt authenticate htpasswd/htdigest

A python snippet to authenticate against a htpasswd hash in Python:
from crypt import crypt
pw = '2600'
htpasswd = 'Jywn1PBEdYjkg'
authd = crypt(pw,htpasswd)==htpasswd
assert authd
authd = crypt('wrongpw',htpasswd)==htpasswd
assert authd==False

Friday, September 10, 2010

freenx and GLX

How to get freenx working with GLX & OpenGL?

me@computer:~$ glxgears
Error: couldn't get an RGB, Double-buffered visual

Try this:

Then things should be just peachy!

Monday, September 6, 2010

distutils/setuptools python install -> automatic .deb generation

I used the python-stdeb in the ubuntu 10.04 repo to install itself using the
newest version from the git repo:


1) Add the following to ~/.pydistutils.cfg:

command-packages: stdeb.command

2) If necessary, edit a stdeb.cfg file which contains the options for debian/control.
The stdeb git repo already has a sensible stdeb.cfg file, but for another project,
the following might be appropriate:

Depends: python-numpy
XS-Python-Version: >= 2.6
Provides: python-myproject

3) python bdist_deb

4) dpkg -i deb_dist/python-stdeb_0.6.0+git-1_all.deb

NB: comparing that .deb
with the one in the
apt cache:

Shows similar install paths ... so I think everythings ok

Another possibility would be:
$ sudo checkinstall --install=no python install

But I suppose stdeb is more savvy in its pythonic ways ...

Thursday, August 19, 2010

anonymous emailing for mailing lists that expose email addresses


import smtplib
SERVER = "your.smtp.server"

FROM = ""
TO = [""] # must be a list

SUBJECT = "my question to the gurus"
TEXT = """

How do I send a message to the mailing list without exposing my email address to 



To hide my email address from spam-bots, this message was sent with a false sender address using Python's smtplib.


# Prepare actual message

message = """\
From: %s
To: %s
Subject: %s

""" % (FROM, ", ".join(TO), SUBJECT, TEXT)

# Send the mail

server = smtplib.SMTP(SERVER)
server.sendmail(FROM, TO, message)

Wednesday, June 30, 2010


Design Patterns: common "Template" solutions to regularly encountered problems/variations-on-that problem. Be careful when learning these that you don't fall victim to "when you have a hammer, everything is a nail". Also learn the Anti-patterns, wikipedia has a good list of anti-patterns.

Algorithms & Data Structures: Analysis, average running time Big O is most important, but understanding worst-case runtime is important too. Designing algorithms vs knowing when to leverage an existing one.

the C++ standard library provides a great many of these, it has a high efficiency sort (from ), it has good collection data structures (vectors, linked lists, maps, etc)

Objected Oriented Analysis And Design: Knowing when to make something an object, when and how to use inheritance and polymorphism, when to not make something an object. Plain old data objects. separation of responsibility: UI is not logic, logic is not UI.

Threading: proper thread synchronization techniques (mutexs, semaphores, conditions, etc), threading patterns such as Producer-Consumer, Inter-process communication

Automata & Computability: (Deterministic|Nondeterministic) Finite State Machines, Regular Languages, Turing Machines

Programming Languages: LL language parsing & rules authoring.

Computer Architecture: Processor design, pipelining, caching, function calling conventions, etc - how to use this knowledge to write more efficient programs

DMG on Linux

Mounting Mac OSX dmg files on Linux

#sudo apt-get install dmg2img
# this outputs myfile.img
dmg2img myfile.dmg

#modprobe hfsplus
sudo mount -t hfsplus -o loop myfile.img $MOUNTDIR

Wednesday, May 12, 2010

Java in Python : JPype on Ubuntu Lucid

sudo apt-get install python-jpype openjdk-6-jdk
sudo update-java-alternatives -s java-6-openjdk

from jpype import *

startJVM("/usr/lib/jvm/java-6-openjdk/jre/lib/amd64/server/", "-ea")

java.lang.System.out.println("Hello World")


KDE okular html links -> firefox

Use firefox instead of konqueror for html links in pdfs viewed in okular:

System settings -> Advanced -> File associations

Type in html in the search field and move firefox above konqueror for all html like content.

Have a lot of fun!

Friday, May 7, 2010

MySQL server on Ubuntu or debian

To setup a mysql server on my ubuntu 9.10 box, I followed this guide:

But connecting from another host gave me the following error:

$ mysql -u mysqluser -p

ERROR 2003 (HY000): Can't connect to MySQL server on '' (111)

The above link indicates this is a network error.

Referring to the above link, I found to enable access I needed to edit /etc/mysql/my.cnf, specifically, change the bind-address from

#bind-address =
bind-address =

and restart mysqld:

$ /etc/init.d/mysql restart

Other useful MySQL links:

Adding users, etc.

MYSQL Docs - Access Privilege System


Thursday, April 29, 2010

delicious tips

URL to search for tags "search" and "android"

Wednesday, April 28, 2010

Solution: Ubuntu upgrade 9.04->9.10 recovery after USB keyboard and mouse lost during package reconfig interaction

This is the most elegant and satisfying fix I experience to avoid a potentially serious problem ...

During the upgrade process from Ubuntu Jaunty 9.04 to Karmic 9.10, The update-manager usually wants to interact with the user to configure packages. At some point during this process, my USB subsystem stopped working, and I could not interact with the dialogs via keyboard or mouse (tried, but PS/2 keyboard did not help. They are generally not hot pluggable). Perhaps one could reboot and resume the process, but this could also leave the system in a bad state. I could get ssh access no problem. Could I get control of the dialogs remotely?

The answer is NX, and the answer is a resounding YES!

I already had a NX server running on the box, so I merely had to follow this HOWTO to enable "desktop shadowing". After editing /usr/NX/etc/server.cfg, enabling Shadowing and restarting the NX server

$ /etc/init.d/nxserver restart

I was able to use the NX client to regain control of the update process. Viola!

For those of you who do not have NX installed already, here is a nice guide. One might have trouble installing the NX server with dpkg, as there would be a dpkg already running due to the upgrade process. In this case, one could install from source.

Another option would be to try and revive the USB subsystem, but this worked so quickly, I didn't need to go down that road.

Monday, April 19, 2010

Rendering code via pygments in your blog

Cofiguring your blog style sheet to include the pygments style by going to Customize->Layout->Edit HTML

Paste in before this line (for example):
/* Footer
----------------------------------------------- */

The following:
.hll { background-color: #ffffcc }
.c { color: #408080; font-style: italic } /* Comment */
.err { border: 1px solid #FF0000 } /* Error */
.k { color: #008000; font-weight: bold } /* Keyword */
.o { color: #666666 } /* Operator */
.cm { color: #408080; font-style: italic } /* Comment.Multiline */
.cp { color: #BC7A00 } /* Comment.Preproc */
.c1 { color: #408080; font-style: italic } /* Comment.Single */
.cs { color: #408080; font-style: italic } /* Comment.Special */
.gd { color: #A00000 } /* Generic.Deleted */
.ge { font-style: italic } /* Generic.Emph */
.gr { color: #FF0000 } /* Generic.Error */
.gh { color: #000080; font-weight: bold } /* Generic.Heading */
.gi { color: #00A000 } /* Generic.Inserted */
.go { color: #808080 } /* Generic.Output */
.gp { color: #000080; font-weight: bold } /* Generic.Prompt */
.gs { font-weight: bold } /* Generic.Strong */
.gu { color: #800080; font-weight: bold } /* Generic.Subheading */
.gt { color: #0040D0 } /* Generic.Traceback */
.kc { color: #008000; font-weight: bold } /* Keyword.Constant */
.kd { color: #008000; font-weight: bold } /* Keyword.Declaration */
.kn { color: #008000; font-weight: bold } /* Keyword.Namespace */
.kp { color: #008000 } /* Keyword.Pseudo */
.kr { color: #008000; font-weight: bold } /* Keyword.Reserved */
.kt { color: #B00040 } /* Keyword.Type */
.m { color: #666666 } /* Literal.Number */
.s { color: #BA2121 } /* Literal.String */
.na { color: #7D9029 } /* Name.Attribute */
.nb { color: #008000 } /* Name.Builtin */
.nc { color: #0000FF; font-weight: bold } /* Name.Class */
.no { color: #880000 } /* Name.Constant */
.nd { color: #AA22FF } /* Name.Decorator */
.ni { color: #999999; font-weight: bold } /* Name.Entity */
.ne { color: #D2413A; font-weight: bold } /* Name.Exception */
.nf { color: #0000FF } /* Name.Function */
.nl { color: #A0A000 } /* Name.Label */
.nn { color: #0000FF; font-weight: bold } /* Name.Namespace */
.nt { color: #008000; font-weight: bold } /* Name.Tag */
.nv { color: #19177C } /* Name.Variable */
.ow { color: #AA22FF; font-weight: bold } /* Operator.Word */
.w { color: #bbbbbb } /* Text.Whitespace */
.mf { color: #666666 } /* Literal.Number.Float */
.mh { color: #666666 } /* Literal.Number.Hex */
.mi { color: #666666 } /* Literal.Number.Integer */
.mo { color: #666666 } /* Literal.Number.Oct */
.sb { color: #BA2121 } /* Literal.String.Backtick */
.sc { color: #BA2121 } /* Literal.String.Char */
.sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */
.s2 { color: #BA2121 } /* Literal.String.Double */
.se { color: #BB6622; font-weight: bold } /* Literal.String.Escape */
.sh { color: #BA2121 } /* Literal.String.Heredoc */
.si { color: #BB6688; font-weight: bold } /* Literal.String.Interpol */
.sx { color: #008000 } /* Literal.String.Other */
.sr { color: #BB6688 } /* Literal.String.Regex */
.s1 { color: #BA2121 } /* Literal.String.Single */
.ss { color: #19177C } /* Literal.String.Symbol */
.bp { color: #008000 } /* Name.Builtin.Pseudo */
.vc { color: #19177C } /* Name.Variable.Class */
.vg { color: #19177C } /* Name.Variable.Global */
.vi { color: #19177C } /* Name.Variable.Instance */
.il { color: #666666 } /* Literal.Number.Integer.Long */

/* end of pygments style sheet */

and save.
Then run your piece of code through pygemtize:
$ pygmentize -f html -o god_writes_good_code.html
Paste the html into the blog post. The preview will not show the colorization, but the blog view will.
import nltk
from nltk.corpus import PlaintextCorpusReader as PtCr

import numpy
import os

def cache_url(url, gunzip=True):
    """fetch the url locally if not already local.
    gunzip - {True,False} - pass through gunzip or not

    filename = os.path.split(url)[-1]
    ext = os.path.splitext(url)[-1]

    # will we be unzipping?
    if gunzip and ext==".gz":
        unzipped = filename[:-3]
        if not os.path.exists(unzipped):
            err = os.system("gunzip -f %s" % filename)
            if err!=0:
                raise OSError, "gunzip error on file: %s" % filename
        return unzipped

Friday, April 16, 2010

Sesame on Tomcat on Ubuntu 9.10

First become root
$ sudo su

Now install JRE

# sudo apt-get install sun-java6-jre sun-java6-plugin sun-java6-fonts libservlet-2.5
# update-alternatives --config java

A few options pop up. I chose this one:


Now tomcat (according to this howto and summarized here)

# apt-get install tomcat6 tomcat6-admin tomcat6-common tomcat6-user tomcat6-docs tomcat6-examples

Try starting tomcat
# /etc/init.d/tomcat6 start
Try stopping
# /etc/init.d/tomcat6 stop
Restart an we're gonna play with it
# /etc/init.d/tomcat6 restart
Query readiness
# /etc/init.d/tomcat6 status
* Tomcat servlet engine is running with pid 3789

Point your browser to http://localhost:8080 and verify that tomcat is up and running.

Now install Sesame on tomcat as follows.

Download the sesame binary package from

Extract it to some dir $SESAME

# cd /var/lib/tomcat6/webapps

Copy the .war files

# cp $SESAME/war/* .

# mkdir -p /usr/share/tomcat6/.aduna/openrdf-sesame/logs
# chown tomcat6:tomcat6 /usr/share/tomcat6/.aduna

Edit /etc/tomcat6/policy.d/50local.policy

and add this stuff:

// =============================================
// Additional permissions for openrdf-workbench
grant codeBase "file:${catalina.base}/webapps/openrdf-workbench/-" {
permission "localhost:8080", "connect";
permission "locahost:8080", "resolve";
// Additional permissions for openrdf-sesame
grant codeBase "file:${catalina.base}/webapps/openrdf-sesame/-" {
permission java.util.PropertyPermission "info.aduna.platform.appdata.basedir", "read";
permission java.util.PropertyPermission "aduna.platform.applicationdata.dir", "read";
permission java.util.PropertyPermission "user.home", "read";
permission java.lang.RuntimePermission "createSecurityManager";
permission java.util.logging.LoggingPermission "control";
permission "/usr/share/tomcat6/.aduna/openrdf-sesame/logs", "read";
permission "/usr/share/tomcat6/.aduna/openrdf-sesame/logs", "write";
permission "/usr/share/tomcat6/.aduna/openrdf-sesame/conf/logback.xml", "read";
permission "/usr/share/tomcat6/.aduna/openrdf-sesame/conf/", "read";
permission java.util.PropertyPermission "*", "read,write";
permission "/usr/share/tomcat6/.aduna/openrdf-sesame/conf", "read,write";

//permission /usr/share/tomcat6/.aduna/openrdf-sesame/*", "read,write";
permission "/usr/share/tomcat6/.aduna/openrdf-sesame/conf/", "read,write";

permission "/usr/share/tomcat6/.aduna/openrdf-sesame/repositories/SYSTEM/", "read,write";

permission "/usr/share/tomcat6/.aduna/openrdf-sesame/repositories/SYSTEM/lock", "read,write";
permission "/usr/share/tomcat6/.aduna/openrdf-sesame/-", "read,write,delete";

permission java.lang.RuntimePermission "shutdownHooks";
permission java.lang.RuntimePermission "getProtectionDomain";
permission java.lang.reflect.ReflectPermission "suppressAccessChecks";

// ===================================================

Create /etc/tomcat6/Catalina/localhost/openrdf-workbench.xml containing:

sudo /etc/init.d/tomcat6 restart



to verify that no Java exceptions are thrown.

Tomcat admin:

emacs /etc/tomcat6/tomcat-users.xml
add user with manager role.

visit http://localhost:8080