So far in this series we have seen how JASON can pass data during processing to an external program allowing some operation to be performed which is not currently available from within JASON. The examples so far have been fairly trivial, however, in this post, I’m going to demonstrate a real application where a processing function for t1 noise reduction was prototyped in python, before being implemented by our development team in the main JASON code.
High dynamic range samples can sometimes show unwanted features in their 2D spectra, parallel to the indirect dimension. This is so-called t1 noise, and arises from small variations in signal intensity between each t1 increment. These variations are not modulated in t1, and hence appear as stripes in the spectra parallel to t1/F1 and related to strong signals in the spectrum. Modern spectrometer hardware largely reduces these artefacts, but it is sometimes necessary to remove them at the data processing stage, hence why we were interested in their removal.
There are a number of algorithms available for the suppression of t1 noise. In this post, we will look at one proposed by Manoleras and Norton and published in the Journal of Biomolecular NMR (J. Biomol. NMR 2 (1992) 485-494). In this approach the average noise in each trace parallel to the t1/F1 axis (i.e. each column) is determined, and then a weighted filter is applied along the t2/F2 dimension (i.e. to each row) using weights derived from the noise profile determined previously.
The Manoleras and Norton algorithm was implemented in a python script, which we will walk through below. Note that this implementation is appropriate only for phase sensitive data.
The script begins with loading some python modules, the h5py library provides functions for accessing the temporary HDF5 files produced JASON and NumPy is a commonly used library for standard mathematical operations on arrays. Following this, a few parameters related to the smoothing algorithm are set. These can be adjusted by the user to control the strength of the denoising, however, these values provide reasonable results and are similar to those used in the original paper.
Next the file passed by JASON is opened and the real part of the spectrum extracted to a python array. Also at this time, the size of the dataset is determined and an empty array in which to store the final results is created.
The Manoleras and Norton algorithm has two parts. The first determines the average noise in each column of the spectrum. This is done by looping over each column of the dataset, removing peaks which are above a certain threshold set by the eta parameter.
The second stage of the algorithm is to apply a smoothing filter to the data with weighting determined by the average noise level found in the previous step. This time, the algorithm loops over each row in the spectrum. The algorithm is such that the maximum value of the smoothing is 1, so aggressive changes to the data are not possible.
Finally, the smoothed data is written back to the temporary HDF5 file and returned to JASON for any final processing to be applied.
The figures below show the same 1H-1H NOESY spectrum of 2-ethyl indanone before and after the application of this filter. The data are plotted at the same contour level.
We have used this script to help implemented a t1 noise reduction algorithm within JASON. This processing item has a few improvements in terms of peak detection and will work with both magnitude mode and phase sensitive datasets. Hopefully this post demonstrates how the simplicity and power of python can be used to rapidly prototype and test new processing functions.
The scripts used in this series of posts are available here and can be used as the basis for your own external commands.