This post explains the concept of image cursors, a new way of accessing image data without the struggle of handling image data types, developed by the Icy team.
When developing plugins for Icy, we repeatedly struggle with handling images of different data types (int, short, long, float, double, etc.) The development team of Icy developed a new way of accessing image data without the struggle of handling image data types. This new way of accessing images is called cursors. They act as if they were pointers to the elements on the image but allow you to ignore the data type of the pixels, providing you with an equivalent double value of the pixel instead. This has two advantages: 1) it simplifies the access to single pixel data and 2) it allows to abstract image processing algorithms from the data types. Additionally, cursors were optimized to be compatible with the caching system of Icy so that memory is handled the best possible way!
In general, there are three types of cursors linked to the way Icy handles images:
- The most general way to access data is through a SequenceCursor , which provides reading and writing access to all the pixels in the images (X, Y, C, Z, T axes).
- Next, the VolumetricImageCursor allows access to 3D image data (X, Y, C, Z axes).
- Finally, planar images (X, Y, C axes) can be accessed through IcyBufferedImageCursor .
In the following paragraphs, each one of these classes will be presented with examples to understand them and get the most out of your images. First, we will start by taking a look at the simplest one, the IcyBufferedImageCursor, then we will add depth to the cursor to get a VolumetricImageCursor, and finally we will handle a time series of volumetric images using SequenceCursor.
IcyBufferedImageCursor
This class provides random pixel access to any pixel on a planar image image, acting like a pointer to a position of the image.
Let’s take the following sample code for a plugin that takes the first image of the active sequence and produces a grayscale image:
import icy.image.IcyBufferedImage;
import icy.image.IcyBufferedImageCursor;
import icy.main.Icy;
import icy.plugin.abstract_.PluginActionable;
import icy.sequence.Sequence;
import icy.util.OMEUtil;
public class IcyBufferedImageTraversal extends PluginActionable
{
@Override
public void run()
{
// 1. Retrieve sequence and first image
Sequence colorSeq = Icy.getMainInterface().getActiveSequence();
IcyBufferedImage colorImg = colorSeq.getFirstImage();
// 2. Create a blank result image
IcyBufferedImage grayImg = new IcyBufferedImage(colorImg.getSizeX(), colorImg.getSizeY(), 1,
colorImg.getDataType_());
// 3. Create cursors for both images
IcyBufferedImageCursor colorImgCursor = new IcyBufferedImageCursor(colorImg);
IcyBufferedImageCursor grayImgCursor = new IcyBufferedImageCursor(grayImg);
// 4. Traverse image
for (int j = 0; j < colorImg.getSizeY(); j++)
{
for (int i = 0; i < colorImg.getSizeX(); i++)
{
double valueSum = 0d;
for (int c = 0; c < colorImg.getSizeC(); c++)
{
// 5. get pixel value at channel c using cursor
valueSum += colorImgCursor.get(i, j, c);
}
// 6. Set pixel value to average of channels
grayImgCursor.setSafe(i, j, 0, valueSum / colorImg.getSizeC());
}
}
// 7. Finish changes on gray image cursor
grayImgCursor.commitChanges();
// 8. Insert gray image in result sequence and display it
Sequence graySeq = new Sequence(OMEUtil.createOMEXMLMetadata(colorSeq.getOMEXMLMetadata()));
graySeq.addImage(grayImg);
Icy.getMainInterface().addSequence(graySeq);
}
}
At first sight, this code has nothing outstanding in it, we simply create a new image and we copy the values from one image to the other. However, by using cursors on our code we have completely forgotten about data types. It won’t matter if the image is of type short or float, our cursor is taking care of that and we only deal with double values. Using cursors you have the freedom of traversing the image in any way you need (X->Y->C or C->X->Y or Y->X->C, etc.).
To use a cursor, we create an instance pointing to the target image (Comment number 3). Then we can move around the image and retrieve values from the image using the get method (Comment number 5). Behind the scenes, this method will perform the appropriate conversions to read the original data type and provide a valid double-precision floating-point value. Similarly, we can also set values on the image using cursors by using either set or setSafe methods (Comment number 6). Using the set method will copy the value provided as parameter without taking into account the interval of valid values of the image data type. This means that an value overflow can occur if the passed value is too large. In order to avoid this, the method setSafe can be used to limit values that are higher than the maximum value of the data type of the target image. Once we are done setting all the values on the target cursor, a call to commitChanges (Comment number 7) is made to ensure the data is correctly set on the target image.
It is important to notice that cursors can be used for both reading and writing data on images. We have used the method setSafe in this case to handle possible value overflows on the destination data type (Comment number 6) .
VolumetricImageCursor
Handling data on a planar stack (volumetric image) using cursors is not very different from using planar image cursors, the only difference is a new parameter when accessing pixel data (the depth). The following example shows the same procedure presented on planar images but using volumetric image cursors.
import icy.image.IcyBufferedImage;
import icy.main.Icy;
import icy.plugin.abstract_.PluginActionable;
import icy.sequence.Sequence;
import icy.sequence.VolumetricImage;
import icy.sequence.VolumetricImageCursor;
import icy.util.OMEUtil;
public class VolumetricImageTraversal extends PluginActionable
{
@Override
public void run()
{
// 1. Retrieve sequence and first volume
Sequence colorSeq = Icy.getMainInterface().getActiveSequence();
VolumetricImage colorVol = colorSeq.getVolumetricImage(0);
// 2. Create a blank result volume
VolumetricImage grayVol = new VolumetricImage();
for (int k = 0; k < colorVol.getSize(); k++)
{
grayVol.setImage(k,
new IcyBufferedImage(colorSeq.getSizeX(), colorSeq.getSizeY(), 1, colorSeq.getDataType_()));
}
// 3. Create cursors for both volumes
VolumetricImageCursor colorVolCursor = new VolumetricImageCursor(colorVol);
VolumetricImageCursor grayVolCursor = new VolumetricImageCursor(grayVol);
// 4. Traverse image
for (int k = 0; k < colorSeq.getSizeZ(); k++) // Z
{
for (int j = 0; j < colorSeq.getSizeY(); j++) // Y
{
for (int i = 0; i < colorSeq.getSizeX(); i++) // X
{
double valueSum = 0d;
for (int c = 0; c < colorSeq.getSizeC(); c++) // C
{
// 5. get pixel value at channel c using cursor
valueSum += colorVolCursor.get(i, j, k, c);
}
// 6. Set pixel value to average of channels
grayVolCursor.setSafe(i, j, k, 0, valueSum / colorSeq.getSizeC());
}
}
}
// 7. Finish changes on gray volume cursor
grayVolCursor.commitChanges();
// 8. Insert gray volume in result sequence and display it
Sequence graySeq = new Sequence(OMEUtil.createOMEXMLMetadata(colorSeq.getOMEXMLMetadata()));
graySeq.addVolumetricImage(0, grayVol);
Icy.getMainInterface().addSequence(graySeq);
}
}
In this example, we first initialize the volume with blank planar images and then set pixel values using a volume cursor (Comment number 2). The volume cursors can be created from either a volumetric image (as in the example at Comment 3) or by specifying one timepoint of a given sequence. The volume traversal is then performed first by depth (Z), then by the Y-axis, then by the X-axis and finally by channel (Comment number 4). Note that the only major change besides using volume cursors instead of image cursors is that we are now traversing the Z-axis of the image with the variable k, which is used in both get and set methods of the cursors.
SequenceImageCursor
Finally we reach to the most general cursor, which allows to traverse entire sequences in any desired order. It behaves similar to both IcyBufferedImageCursor and VolumetricImageCursor but handles, in addition to the planar image and the image depth, the time axis. The following example shows the same procedure presented in the previous two codes but it is applied to a sequence.
import icy.image.IcyBufferedImage;
import icy.main.Icy;
import icy.plugin.abstract_.PluginActionable;
import icy.sequence.Sequence;
import icy.sequence.SequenceCursor;
import icy.util.OMEUtil;
public class SequenceTraversal extends PluginActionable
{
@Override
public void run()
{
// 1. Retrieve active sequence
Sequence colorSeq = Icy.getMainInterface().getActiveSequence();
// 2. Create a blank result sequence using the same input image metadata
Sequence graySeq = new Sequence(OMEUtil.createOMEXMLMetadata(colorSeq.getOMEXMLMetadata()));
for (int l = 0; l < colorSeq.getSizeT(); l++) // T
{
for (int k = 0; k < colorSeq.getSizeZ(); k++) // Z
{
graySeq.addImage(k,
new IcyBufferedImage(colorSeq.getSizeX(), colorSeq.getSizeY(), 1, colorSeq.getDataType_()));
}
}
// 3. Create cursors for both volumes
SequenceCursor colorSeqCursor = new SequenceCursor(colorSeq);
SequenceCursor graySeqCursor = new SequenceCursor(graySeq);
// 4. Traverse sequence
for (int l = 0; l < colorSeq.getSizeT(); l++) // T
{
for (int k = 0; k < colorSeq.getSizeZ(); k++) // Z
{
for (int j = 0; j < colorSeq.getSizeY(); j++) // Y
{
for (int i = 0; i < colorSeq.getSizeX(); i++) // X
{
double valueSum = 0d;
for (int c = 0; c < colorSeq.getSizeC(); c++) // C
{
// 5. get pixel value at channel c using cursor
valueSum += colorSeqCursor.get(i, j, k, l, c);
}
// 6. Set pixel value to average of channels
graySeqCursor.setSafe(i, j, k, l, 0, valueSum / colorSeq.getSizeC());
}
}
}
}
// 7. Finish changes on gray sequence cursor
graySeqCursor.commitChanges();
// 8. Display result
Icy.getMainInterface().addSequence(graySeq);
}
}
We hope the examples presented in this post are helpful for you to understand how cursors work and also that it serves as a reference for future developers in need of data type abstraction when traversing sequences in Icy.