mpastell.com - Articles (Atom Feed)

Create odf, pdf and html report from a single Sweave document

2010-03-24T22:00:00Z

A lot of us know about Sweave and Latex and they work very well in creating elegant dynamic reports from R computation. However, sometimes we would like to also produce a word processing document for a colleague or a html version of the same report. Now there are tools for producing these like odfWeave. But wouldn't it be nice to get all of the output formats from the source document with same mark up?

Today I'm going to write about a good option how to get a nice looking odf, pdf and html report from the same source: ascii package with reStructuredText (reST) markup. reStructuredText is a simple plain text mark up that can be converted to several formats using the Python Docutils. It is simpler to write than Latex and it also the syntax that is used in Sphinx documents. The ascii package has a Sweave driver for weaving R code from documents written with reST and some other mark up languages as well. I have created a simple example document ascii-example.Rnw, which demonstrates the capabilities of the combination. The source file is a Sweave document with reST mark up instead of Latex. I had to make small modifications to the in ReST driver in ascii package to get this example to work¹ (download it here: newRest.R). I then processed the example with the new driver in R:

> library(ascii)
> source('newRest.R')
> ReST('ascii-example.Rnw')

Weaving produced this ascii-example.rst reST document which I then converted to odf, html, pdf with pdflatex and pdf with rst2pdf using:

rst2odt ascii-example.rst ascii-example.odt
rst2html ascii-example.rst ascii-example.html
rst2latex ascii-example.rst ascii-example.tex && pdflatex ascii-example.tex
rst2pdf ascii-example.rst

And here are the results: odt, html, pdf from latex and pdf from rst2pdf. You'll need to have Docutils and odtwriter for the conversion.

I have used the default options, but there are a lot options that can be passed to the docutils writers to customize the output, such as custom stylesheets. The image format in the pdfs is not optimal, but in can be changed with the "res" option in code chunk or you can also choose to use pdf format instead of jpg, but then it won't show up in html documents. I think using reST with ascii package is a good option for producing reports and tutorials in multiple output formats. That is also because I'm already familiar with reST directives and also use Sphinx for other purposes too. The reST syntax is also very fast to learn and I think it is definitely worth exploring. I also recommend using Sphinx if you only want to get html and pdf output, because it has more directives than plain reST e.g. math support.

I changed the driver to output code chunks in reST literal environment :: instead of .. codeblock:: r, because the codeblock directive is not supported in odt conversion and very poorly supported in latex conversion. ↩

Pweave - Sweave for Python

2010-03-02T22:00:00Z

After looking into different options for literate programming with Python I ended up writing my own tool and call it Pweave. I have tried to implement the functionality of Sweave, that I really like and use for my R projects. Pweave is a single python script that is able to weave a python code between <<>>= and @ blocks and include the results in the document. It can be used with reStructured text and Latex documents.

Update:. I've set up a Pweave website, you can also download it here.

Codeblock options Pweave currently allows following options for the codeblocks (e.g. <>=): fig True or False. Whether a matplotlib plot produced in the codeblock should be included in the file. echo True or False. Echo the python code in the codeblock? eval True or False. Should the codeblock be evaluated? results "rst", "tex" or "verbatim". The output format of the printed results. Example: The syntax of Pweave is similar to Sweave, here is a small example sourcefile ma.Pnw which can be processed with Pweave using:

Pweave ma.Pnw

which produces a reStructured text document ma.rst, which can in turn be processed to ma.pdf with rst2latex and pdflatex.

Python in Sweave document

2010-02-08T22:00:00Z

Modifications to the custom driver:
Example usage

Lately I have been using a lot of Python for signal processing and I quite like SciPy. However, I have been missing something like Sweave, which is great literate programming environment for R. Today I managed to look a bit more into it and found this hack on how to use Python code in Sweave document by Romain Francois. However, I wanted to be able to catch the input as well as output so I made a small changes to his Sweave Driver.

Modifications to the custom driver:

driver <- RweaveLatex() 
runcode <- driver$runcode

driver$runcode <- function(object, chunk, options){ 
  if( options$engine == "python")
    { driver$writedoc( object, c("\\\\begin{verbatim}", chunk,
    "\\\\end{verbatim}", "\\\\begin{python}", chunk,"\\\\end{python}") ) }
  else{ runcode( object, chunk, options ) 
  } 
}

The driver now catches the input in verbatim environment (which can be easily changed to listings) and the output to python environment. The tex document can then be processed with the python latex package to evaluate the python expressions. Tho use the driver you need to put the option "engine="python" in your code chunks.

Example usage

My example is python code that calculates and plots the frequency response of a moving average filter. Here is the code in the Sweave document ma.Rnw. Process the file in R using the custom driver above:

Sweave( "ma.Rnw", driver = driver)

It should produce ma.tex. Then run latex (make sure you have the python package installed):

pdflatex -shell-escape ma.tex

Which should then in turn output this ma.pdf file.

FIR filter design with Python and SciPy

2010-01-17T22:00:00Z

Lowpass FIR filter
Highpass FIR Filter
Bandpass FIR filter
Functions for frequency, phase, impulse and step response

SciPy really has good capabilities for DSP, but the filter design functions lack good examples. A while back I wrote about IIR filter design with SciPy. Today I'm going to implement lowpass, highpass and bandpass example for FIR filters. We use the same functions (mfreqz and impz, shown in the end of this post as well) as in the previous post to get the frequency, phase, impulse and step responses. You can download this example along with the functions here FIR_design.py. To begin with we'll import pylab and scipy.signal:

from pylab import *
import scipy.signal as signal

Lowpass FIR filter

Designing a lowpass FIR filter is very simple to do with SciPy, all you need to do is to define the window length, cut off frequency and the window:

n = 61
a = signal.firwin(n, cutoff = 0.3, window = "hamming")
#Frequency and phase response
mfreqz(a)
show()
#Impulse and step response
figure(2)
impz(a)
show()

Which yields:

Highpass FIR Filter

SciPy does not have a function for directly designing a highpass FIR filter, however it is fairly easy design a lowpass filter and use spectral inversion to convert it to highpass. See e.g Chp 16 of The Scientist and Engineer's Guide to Digital Signal Processing for the theory, the last page has an example code.

n = 101
a = signal.firwin(n, cutoff = 0.3, window = "hanning")
#Spectral inversion
a = -a
a[n/2] = a[n/2] + 1
mfreqz(a)
show()

Bandpass FIR filter

To get a bandpass FIR filter with SciPy we first need to design appropriate lowpass and highpass filters and then combine them:

n = 1001
#Lowpass filter
a = signal.firwin(n, cutoff = 0.3, window = 'blackmanharris')
#Highpass filter with spectral inversion
b = - signal.firwin(n, cutoff = 0.5, window = 'blackmanharris'); b[n/2] = b[n/2] + 1
#Combine into a bandpass filter
d = - (a+b); d[n/2] = d[n/2] + 1
#Frequency response
mfreqz(d)
show()

Functions for frequency, phase, impulse and step response

#Plot frequency and phase response
def mfreqz(b,a=1):
    w,h = signal.freqz(b,a)
    h_dB = 20 * log10 (abs(h))
    subplot(211)
    plot(w/max(w),h_dB)
    ylim(-150, 5)
    ylabel('Magnitude (db)')
    xlabel(r'Normalized Frequency (x$\pi$rad/sample)')
    title(r'Frequency response')
    subplot(212)
    h_Phase = unwrap(arctan2(imag(h),real(h)))
    plot(w/max(w),h_Phase)
    ylabel('Phase (radians)')
    xlabel(r'Normalized Frequency (x$\pi$rad/sample)')
    title(r'Phase response')
    subplots_adjust(hspace=0.5)

#Plot step and impulse response
def impz(b,a=1):
    l = len(b)
    impulse = repeat(0.,l); impulse[0] =1.
    x = arange(0,l)
    response = signal.lfilter(b,a,impulse)
    subplot(211)
    stem(x, response)
    ylabel('Amplitude')
    xlabel(r'n (samples)')
    title(r'Impulse response')
    subplot(212)
    step = cumsum(response)
    stem(x, step)
    ylabel('Amplitude')
    xlabel(r'n (samples)')
    title(r'Step response')
    subplots_adjust(hspace=0.5)

ESS on Mac OS X

2009-11-24T22:00:00Z

One of the search terms that bring people frequently to my site is "install ESS on Mac OS X" or something like that. As it turns out installing ESS on OS X is really easy, but Google search does not really bring up good instructions. There are at least two easy options:

Use Aquamacs, it comes bundled with ESS
Use Carbon Emacs, it also comes with ESS, but you need to add the line (require 'ess-site) to your .emacs file in your home directory in order to enable it.

That's it!