August 17th, 2009
Today is officially “pencils-down” day for the Google Summer of Code Project 2009. I have been a mentor this year for the Apache Commons-Net SSH project, which aims to add SSH and SCP support to Commons-Net.
The project has been a great success, mainly down to the super work performed by the student, Shikhar, who has put in tremendous work to get a fully functional SSH/SCP client built (with thanks to the efforts of the Mina SSHD project, whose codebase we originally based the effort on). All of the goals for the project have been ticked, and some extra ones accomplished too.
I had great help and input from Chico, another Apache committer, throughout the project, and so it’s been a great experience all round. This will form the basis of a release, but for now, the code is hosted on googlecode at: http://code.google.com/p/commons-net-ssh/
Posted in Coding | No Comments »
August 17th, 2009
Running R on a Linux server in headless mode (i.e. producing graphics without XWindows running) can be tricky. Some people recommend using a virtual X framebuffer. However, I’ve found that the best approach (at least im my opinion) is to use the R interface to Cairo. This allows R to produce png graphics in headless mode, and also produces very nice looking graphs. I configured R as follows (after downloading and building pixman-0.15.18, and cairo-1.8.8:
./configure --with-gnu-ld --with-x --with-cairo
This will produce an R binary with cairo support that can be run non-interactively and produce graphical output – very useful for running automated statistical reports.
You can check that Cairo support is enabled by checking the return value of the capabilities() function:
> capabilities()
jpeg png tiff tcltk X11 aqua http/ftp sockets
TRUE TRUE FALSE FALSE FALSE FALSE TRUE TRUE
libxml fifo cledit iconv NLS profmem cairo
TRUE TRUE TRUE TRUE TRUE FALSE TRUE
Finally, some notes on connecting X11 clients using Cygwin (which I always forget how to do). On the server, check /etc/ssh/sshd_config for the line
X11Forwarding yes
And then run a local X server:
XWin -clipboard -emulate3buttons -multiwindow
Once this is running, from an xterm, run ssh, passing in the -X argument to enable X forwarding.
ssh -X -l username myserver
X11-based applications can then be run from this session.
Posted in Coding, R | No Comments »
July 15th, 2009
I have been playing with Lilypond a little more, and here are some themes that I have transcribed. They are all from John Carpenter movies: the first is the theme from “Halloween”, which is an interesting 5/4 time piano riff; the second is the synth theme from “Escape From New York”, and the final is the ending credits music of “The Thing” (known as “Humanity Part II” on the soundtrack album). The last one is a bit messy as I couldnt get Lilypond to hold the ties across multiple bars properly.
carpenter.pdf
This is still not 100% finished – there are some omissions. Plus, my transcription may be incorrect in places.
Here is the PDF:
carpenter.pdf
And the Lilypond source:
carpenter
Posted in Music | No Comments »
July 15th, 2009
Last week I presented a short talk at the 2009 UseR conference. The conference was the usual mix of varied topics (even more varied than usual this year) and a lot of interesting discussions.
Here are the slides.
Incidentally, Mango Solutions have set up a website for the London R user group meetings here: http://www.londonr.org/
Posted in Coding | No Comments »
July 14th, 2009
I have been playing with the kdb/R interface from kx.com, and had some problems installing with Cygwin gcc. It may be possible to get this to work with Cygwin gcc + a Win32 threads library, but in the meantime I installed MinGW, and it works perfectly. Here are the steps (basically as per the kx docs):
1. Download c.o from here: http://kx.com/q/w32/
2. gcc -c base.c -I. -I "${R_HOME}/include/"
3. gcc -Wl,--export-all-symbols -shared -o qserver.dll c.o base.o ${r
-HOME}/bin/R.dll -lws2_32
The resulting qserver.dll can be loaded via dyn.load(), and then (just using the qserver.R supplied by kx) from within R:
source("qserver.R")
conn < - open_connection("server", 12345)
result <- execute(conn, "select avg bid by sym from fx_quote")
x <- as.data.frame(mapply(FUN=c, result))
> head(x, 10)
V1 V2
1 AUD= 0.792402880224811
2 AUD=D2 0.791632149468651
3 AUD=EBS 0.790402776387278
4 AUDCHF=R 0.85955071021153
5 AUDJPY=R 75.0707755671935
6 BRL= 1.97194091379422
7 CAD= 1.15980648929715
8 CAD=D2 1.15962545479939
9 CAD=EBS 1.14104373919176
10 CADJPY=R 81.6389284332255
Posted in Coding | No Comments »
July 2nd, 2009
Here is a handy way to get awk to preprocess a line and add a timestamp (Put here as I will probably forget how to do this straight away again!)
echo "foo,bar" | awk '{x="'"`date +%Y%M%d%S%N`"'"; printf "%s,%s\n",x,$0 }'
Posted in Coding | No Comments »
April 2nd, 2009
The inaugural London UseR event was a great success, with a lot of interesting people and a very constructive networking atmosphere!
I gave a (slightly disjointed) talk on concurrency and the bigmemory package in R (more on that later this year at UseR! 2009 in France).
The slides are here.
Posted in Coding, R | No Comments »
March 5th, 2009
On Tuesday March 31st, Mango Solutions are sponsoring the inaugural London R User Group Meeting. It will be a great opportunity to meet other R users and find out how people are using it. As the first one of its kind in London, I would expect a high level of interest. There will be a number of speakers presenting on various topics, using the UseR! panel format (short talks at around 15 minutes or so). I will be giving a short presentation, most likely following on from the real-time data integration work I presented on at UseR! 2008.
See this page for details on the event:
http://www.mango-solutions.com/events/UseRLondon.html
Posted in R | No Comments »
March 3rd, 2009
Problem 28 on the Project Euler website asks what is the sum of both diagonals in a 1001×1001 clockwise spiral. This was an interesting one: the relationship between the numbers on the diagonals is easy to deduce, but expressing it succinctly in R took a little bit of tweaking. I’m sure it could be compressed even further.
# Problem 28
spiral.size <- function(n) {
stopifnot(n %% 2 ==1)
if (n==1) {
return(1)
}
sum(cumsum(rep(2*seq(1,floor(n/2)), rep(4,floor(n/2))))+1)+1
}
spiral.size(1001)
Posted in Coding, Project Euler, R | No Comments »
March 1st, 2009
Problem 22 on Project Euler proves a text file containing a large number of comma-delimited names and asks us to calculate the numeric sum of the alphabetical score for each name multiplied by the name’s position in the original list. This is made slightly easier by the presence of the predefined LETTERS variable in R.
problem22 <- function() {
namelist <- scan(file="c:/temp/names.txt", sep=",", what="", na.strings="")
sum(unlist(
lapply(namelist,
function(Z) which(namelist==Z) * sum(match(unlist(strsplit(Z,"")), LETTERS)))))
}
Posted in Coding, Project Euler, R | No Comments »
