LaTeX and git

Hi,

Some of you may remember our discussion at dinner about using git/SVN with LaTeX documents. One of the problems that was mentioned was line-based diff - whenever the formatting of the paragraph changes all of the lines are marked as modified, even if the content did not change.

* git diff --color-words

I found very nice solution to the problem: in git diff you can use the option --color-words which performs word-based diff and highlights the changes in colors.

* gitattributes

You can also put the following line into your .gitattributes file:

*.tex diff=tex

which treats some tags (likes \section{}) as separate words even if there is no whitespace in between.

I am almost converted to git - thanks for opening my eyes ;)

Cheers,

Bartek

PyAS - nuggets

PyAS - Nuggets

Ipython cpaste magic func to paste in code.

cython -a -> annotated html with colours for how long things take ... click to expand

Git & github getting started

Global setup:

Download and install Git
git config --global user.name "Your Name"
git config --global user.email eilif@gmx.de


Next steps:

mkdir test
cd test
git init
touch README
git add README
git commit -m 'first commit'
git remote add origin git@github.com:markovg/test.git
git push origin master


Existing Git Repo?

cd existing_git_repo
git remote add origin git@github.com:markovg/test.git
git push origin master


Importing a Subversion Repo?

Click here


When you're done:

Continue

Emacs tricks: cua-mode

cua-mode:

You cut some code from a website and it has a few columns of junk at the front, enter cua-mode

M-x cua-mode

Set marker, C-RET enters column mode. Select the block, C-x to cut.

python crypt authenticate htpasswd/htdigest

A python snippet to authenticate against a htpasswd hash in Python:

from crypt import crypt
pw = '2600'
htpasswd = 'Jywn1PBEdYjkg'
authd = crypt(pw,htpasswd)==htpasswd
assert authd
authd = crypt('wrongpw',htpasswd)==htpasswd
assert authd==False

freenx and GLX

How to get freenx working with GLX & OpenGL?

me@computer:~$ glxgears
Error: couldn't get an RGB, Double-buffered visual

Try this:
$ export LIBGL_ALWAYS_INDIRECT=1

Then things should be just peachy!

distutils/setuptools python setup.py install -> automatic .deb generation

I used the python-stdeb in the ubuntu 10.04 repo to install itself using the
newest version from the git repo:

http://github.com/astraw/stdeb.git

Steps:

1) Add the following to ~/.pydistutils.cfg:

vvvvvvvvvvvvvvvv
[global]
command-packages: stdeb.command
^^^^^^^^^^^^^^

2) If necessary, edit a stdeb.cfg file which contains the options for debian/control.
The stdeb git repo already has a sensible stdeb.cfg file, but for another project,
the following might be appropriate:

[DEFAULT]
Depends: python-numpy
XS-Python-Version: >= 2.6
Provides: python-myproject


3) python setup.py bdist_deb

4) dpkg -i deb_dist/python-stdeb_0.6.0+git-1_all.deb

NB: comparing that .deb
with the one in the
apt cache:
/var/cache/apt/archives/python-stdeb_0.5.1-1_all.deb

Shows similar install paths ... so I think everythings ok

NB:
Another possibility would be:
$ sudo checkinstall --install=no python setup.py install

But I suppose stdeb is more savvy in its pythonic ways ...

anonymous emailing for mailing lists that expose email addresses

#!/usr/bin/python

import smtplib
SERVER = "your.smtp.server"

FROM = "joe@example.com"
TO = ["mailinglist@exposes.emails.com"] # must be a list


SUBJECT = "my question to the gurus"
TEXT = """

How do I send a message to the mailing list without exposing my email address to 
spambots?

cheers,

j.


----
To hide my email address from spam-bots, this message was sent with a false sender address using Python's smtplib.

"""

# Prepare actual message

message = """\
From: %s
To: %s
Subject: %s

%s
""" % (FROM, ", ".join(TO), SUBJECT, TEXT)

# Send the mail

server = smtplib.SMTP(SERVER)
server.sendmail(FROM, TO, message)
server.quit()

self_taught_programmers_need_these_things.txt

Design Patterns: common "Template" solutions to regularly encountered problems/variations-on-that problem. Be careful when learning these that you don't fall victim to "when you have a hammer, everything is a nail". Also learn the Anti-patterns, wikipedia has a good list of anti-patterns.

Algorithms & Data Structures: Analysis, average running time Big O is most important, but understanding worst-case runtime is important too. Designing algorithms vs knowing when to leverage an existing one.

the C++ standard library provides a great many of these, it has a high efficiency sort (from ), it has good collection data structures (vectors, linked lists, maps, etc)

Objected Oriented Analysis And Design: Knowing when to make something an object, when and how to use inheritance and polymorphism, when to not make something an object. Plain old data objects. separation of responsibility: UI is not logic, logic is not UI.

Threading: proper thread synchronization techniques (mutexs, semaphores, conditions, etc), threading patterns such as Producer-Consumer, Inter-process communication

Automata & Computability: (Deterministic|Nondeterministic) Finite State Machines, Regular Languages, Turing Machines

Programming Languages: LL language parsing & rules authoring.

Computer Architecture: Processor design, pipelining, caching, function calling conventions, etc - how to use this knowledge to write more efficient programs

DMG on Linux

Mounting Mac OSX dmg files on Linux

#sudo apt-get install dmg2img
# this outputs myfile.img
dmg2img myfile.dmg

#modprobe hfsplus
sudo mount -t hfsplus -o loop myfile.img $MOUNTDIR

Java in Python : JPype on Ubuntu Lucid

sudo apt-get install python-jpype openjdk-6-jdk
sudo update-java-alternatives -s java-6-openjdk

from jpype import *

startJVM("/usr/lib/jvm/java-6-openjdk/jre/lib/amd64/server/libjvm.so", "-ea")

java.lang.System.out.println("Hello World")

shutdownJVM()

KDE okular html links -> firefox

Use firefox instead of konqueror for html links in pdfs viewed in okular:

System settings -> Advanced -> File associations

Type in html in the search field and move firefox above konqueror for all html like content.

Have a lot of fun!

MySQL server on Ubuntu or debian

To setup a mysql server on my ubuntu 9.10 box, I followed this guide:

http://library.linode.com/databases/mysql/ubuntu-9.10-karmic

But connecting from another host gave me the following error:

$ mysql -u mysqluser -p --host=myhost.com

ERROR 2003 (HY000): Can't connect to MySQL server on 'myhost.com' (111)

http://dev.mysql.com/doc/refman/5.1/en/access-denied.html

The above link indicates this is a network error.

http://www.aboutdebian.com/database.htm

Referring to the above link, I found to enable access I needed to edit /etc/mysql/my.cnf, specifically, change the bind-address from 127.0.0.1:

#bind-address = 127.0.0.1
bind-address = myhost.com

and restart mysqld:

$ /etc/init.d/mysql restart


Other useful MySQL links:

Adding users, etc.

MYSQL Docs - Access Privilege System

MySQL and SSL

delicious tips

URL to search for tags "search" and "android"

http://delicious.com/mydelcious_account/search+android

Solution: Ubuntu upgrade 9.04->9.10 recovery after USB keyboard and mouse lost during package reconfig interaction

This is the most elegant and satisfying fix I experience to avoid a potentially serious problem ...

During the upgrade process from Ubuntu Jaunty 9.04 to Karmic 9.10, The update-manager usually wants to interact with the user to configure packages. At some point during this process, my USB subsystem stopped working, and I could not interact with the dialogs via keyboard or mouse (tried, but PS/2 keyboard did not help. They are generally not hot pluggable). Perhaps one could reboot and resume the process, but this could also leave the system in a bad state. I could get ssh access no problem. Could I get control of the dialogs remotely?

The answer is NX, and the answer is a resounding YES!

I already had a NX server running on the box, so I merely had to follow this HOWTO to enable "desktop shadowing". After editing /usr/NX/etc/server.cfg, enabling Shadowing and restarting the NX server

$ /etc/init.d/nxserver restart

I was able to use the NX client to regain control of the update process. Viola!

For those of you who do not have NX installed already, here is a nice guide. One might have trouble installing the NX server with dpkg, as there would be a dpkg already running due to the upgrade process. In this case, one could install from source.

Another option would be to try and revive the USB subsystem, but this worked so quickly, I didn't need to go down that road.

Rendering code via pygments in your blog

Cofiguring your blog style sheet to include the pygments style by going to Customize->Layout->Edit HTML

Paste in before this line (for example):
/* Footer
----------------------------------------------- */
The following:

.hll { background-color: #ffffcc }
.c { color: #408080; font-style: italic } /* Comment */
.err { border: 1px solid #FF0000 } /* Error */
.k { color: #008000; font-weight: bold } /* Keyword */
.o { color: #666666 } /* Operator */
.cm { color: #408080; font-style: italic } /* Comment.Multiline */
.cp { color: #BC7A00 } /* Comment.Preproc */
.c1 { color: #408080; font-style: italic } /* Comment.Single */
.cs { color: #408080; font-style: italic } /* Comment.Special */
.gd { color: #A00000 } /* Generic.Deleted */
.ge { font-style: italic } /* Generic.Emph */
.gr { color: #FF0000 } /* Generic.Error */
.gh { color: #000080; font-weight: bold } /* Generic.Heading */
.gi { color: #00A000 } /* Generic.Inserted */
.go { color: #808080 } /* Generic.Output */
.gp { color: #000080; font-weight: bold } /* Generic.Prompt */
.gs { font-weight: bold } /* Generic.Strong */
.gu { color: #800080; font-weight: bold } /* Generic.Subheading */
.gt { color: #0040D0 } /* Generic.Traceback */
.kc { color: #008000; font-weight: bold } /* Keyword.Constant */
.kd { color: #008000; font-weight: bold } /* Keyword.Declaration */
.kn { color: #008000; font-weight: bold } /* Keyword.Namespace */
.kp { color: #008000 } /* Keyword.Pseudo */
.kr { color: #008000; font-weight: bold } /* Keyword.Reserved */
.kt { color: #B00040 } /* Keyword.Type */
.m { color: #666666 } /* Literal.Number */
.s { color: #BA2121 } /* Literal.String */
.na { color: #7D9029 } /* Name.Attribute */
.nb { color: #008000 } /* Name.Builtin */
.nc { color: #0000FF; font-weight: bold } /* Name.Class */
.no { color: #880000 } /* Name.Constant */
.nd { color: #AA22FF } /* Name.Decorator */
.ni { color: #999999; font-weight: bold } /* Name.Entity */
.ne { color: #D2413A; font-weight: bold } /* Name.Exception */
.nf { color: #0000FF } /* Name.Function */
.nl { color: #A0A000 } /* Name.Label */
.nn { color: #0000FF; font-weight: bold } /* Name.Namespace */
.nt { color: #008000; font-weight: bold } /* Name.Tag */
.nv { color: #19177C } /* Name.Variable */
.ow { color: #AA22FF; font-weight: bold } /* Operator.Word */
.w { color: #bbbbbb } /* Text.Whitespace */
.mf { color: #666666 } /* Literal.Number.Float */
.mh { color: #666666 } /* Literal.Number.Hex */
.mi { color: #666666 } /* Literal.Number.Integer */
.mo { color: #666666 } /* Literal.Number.Oct */
.sb { color: #BA2121 } /* Literal.String.Backtick */
.sc { color: #BA2121 } /* Literal.String.Char */
.sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */
.s2 { color: #BA2121 } /* Literal.String.Double */
.se { color: #BB6622; font-weight: bold } /* Literal.String.Escape */
.sh { color: #BA2121 } /* Literal.String.Heredoc */
.si { color: #BB6688; font-weight: bold } /* Literal.String.Interpol */
.sx { color: #008000 } /* Literal.String.Other */
.sr { color: #BB6688 } /* Literal.String.Regex */
.s1 { color: #BA2121 } /* Literal.String.Single */
.ss { color: #19177C } /* Literal.String.Symbol */
.bp { color: #008000 } /* Name.Builtin.Pseudo */
.vc { color: #19177C } /* Name.Variable.Class */
.vg { color: #19177C } /* Name.Variable.Global */
.vi { color: #19177C } /* Name.Variable.Instance */
.il { color: #666666 } /* Literal.Number.Integer.Long */

/* end of pygments style sheet */

and save.
Then run your piece of code through pygemtize:
$ pygmentize -f html -o god_writes_good_code.html god_writes_good_code.py
Paste the html into the blog post. The preview will not show the colorization, but the blog view will.
Example:

import nltk
from nltk.corpus import PlaintextCorpusReader as PtCr

import numpy
import os


def cache_url(url, gunzip=True):
    """fetch the url locally if not already local.
    gunzip - {True,False} - pass through gunzip or not
    """

    filename = os.path.split(url)[-1]
    ext = os.path.splitext(url)[-1]

    # will we be unzipping?
    if gunzip and ext==".gz":
        unzipped = filename[:-3]
        if not os.path.exists(unzipped):
            err = os.system("gunzip -f %s" % filename)
            if err!=0:
                raise OSError, "gunzip error on file: %s" % filename
    
        return unzipped

Sesame on Tomcat on Ubuntu 9.10

First become root
$ sudo su

Now install JRE

# sudo apt-get install sun-java6-jre sun-java6-plugin sun-java6-fonts libservlet-2.5
# update-alternatives --config java

A few options pop up. I chose this one:

/usr/lib/jvm/java-6-sun/jre/bin/java

Now tomcat (according to this howto and summarized here)

# apt-get install tomcat6 tomcat6-admin tomcat6-common tomcat6-user tomcat6-docs tomcat6-examples

Try starting tomcat
# /etc/init.d/tomcat6 start
Try stopping
# /etc/init.d/tomcat6 stop
Restart an we're gonna play with it
# /etc/init.d/tomcat6 restart
Query readiness
# /etc/init.d/tomcat6 status
* Tomcat servlet engine is running with pid 3789

Point your browser to http://localhost:8080 and verify that tomcat is up and running.

Now install Sesame on tomcat as follows.

Download the sesame binary package from http://www.openrdf.org/download.jsp.

Extract it to some dir $SESAME

# cd /var/lib/tomcat6/webapps

Copy the .war files

# cp $SESAME/war/* .



# mkdir -p /usr/share/tomcat6/.aduna/openrdf-sesame/logs
# chown tomcat6:tomcat6 /usr/share/tomcat6/.aduna

Edit /etc/tomcat6/policy.d/50local.policy

and add this stuff:

// =============================================
// Additional permissions for openrdf-workbench
grant codeBase "file:${catalina.base}/webapps/openrdf-workbench/-" {
permission java.net.SocketPermission "localhost:8080", "connect";
permission java.net.SocketPermission "locahost:8080", "resolve";
};
//
// Additional permissions for openrdf-sesame
grant codeBase "file:${catalina.base}/webapps/openrdf-sesame/-" {
permission java.util.PropertyPermission "info.aduna.platform.appdata.basedir", "read";
permission java.util.PropertyPermission "aduna.platform.applicationdata.dir", "read";
permission java.util.PropertyPermission "user.home", "read";
//permission java.security.AllPermission;
permission java.lang.RuntimePermission "createSecurityManager";
permission java.util.logging.LoggingPermission "control";
permission java.io.FilePermission "/usr/share/tomcat6/.aduna/openrdf-sesame/logs", "read";
permission java.io.FilePermission "/usr/share/tomcat6/.aduna/openrdf-sesame/logs", "write";
permission java.io.FilePermission "/usr/share/tomcat6/.aduna/openrdf-sesame/conf/logback.xml", "read";
permission java.io.FilePermission "/usr/share/tomcat6/.aduna/openrdf-sesame/conf/proxy.properties", "read";
permission java.util.PropertyPermission "*", "read,write";
permission java.io.FilePermission "/usr/share/tomcat6/.aduna/openrdf-sesame/conf", "read,write";

//permission java.io.FilePermission /usr/share/tomcat6/.aduna/openrdf-sesame/*", "read,write";
permission java.io.FilePermission "/usr/share/tomcat6/.aduna/openrdf-sesame/conf/proxy.properties.default", "read,write";

permission java.io.FilePermission "/usr/share/tomcat6/.aduna/openrdf-sesame/repositories/SYSTEM/memorystore.data", "read,write";

permission java.io.FilePermission "/usr/share/tomcat6/.aduna/openrdf-sesame/repositories/SYSTEM/lock", "read,write";
permission java.io.FilePermission "/usr/share/tomcat6/.aduna/openrdf-sesame/-", "read,write,delete";

permission java.lang.RuntimePermission "shutdownHooks";
permission java.lang.RuntimePermission "getProtectionDomain";
permission java.lang.reflect.ReflectPermission "suppressAccessChecks";

};
// ===================================================



Create /etc/tomcat6/Catalina/localhost/openrdf-workbench.xml containing:





sudo /etc/init.d/tomcat6 restart



Visit:

http://localhost:8080/openrdf-workbench
http://localhost:8080/openrdf-sesame

to verify that no Java exceptions are thrown.

Tomcat admin:

emacs /etc/tomcat6/tomcat-users.xml
add user with manager role.

visit http://localhost:8080