ROI statistics – workbook to file size limit in text format?

Home › Forums › General Issues › ROI statistics – workbook to file size limit in text format?

Tagged: 

  •
    deborakeller0@gmail.com November 8, 2019 at 9:53 am

    Hi,

    I am having issues exporting the spot stats from my spot detector as it seems I am getting to the size limit of both workbook and text export…
    I have 98529 spots detected (over 193 frames on several nuclei per frame – already taking only spots >4 size)… picture in the next post

    any solution to bypass this? the protocol goes through all the analysis but then does not write the data to the file which is a bit sad and I can’t use the rest of the analysis.

    I’d rather not crop if I can avoid it (because of issues later on finding the ringht cells etc).
    but is this something solvable?
    Cheers

    Debbi

     
    deborakeller0@gmail.com November 8, 2019 at 9:54 am

    files where spot stats could not get exported are 1kb

    Stephane Dallongeville November 8, 2019 at 4:41 pm

    Hi Debbi,

    XLSX should be able to handle more than 1 000 000 rows while XLS is limited to 65536 rows maximum.
    ROI Statistics export in XLSX format by default, so i don’t understand why it doesn’t work in your case.

    Does the problem appears with the ROI Statistics and ROI inclusion analysis plugin or only with the latter ?
    In the meantime I will try to replicate the issue.

    Best,

    – Stephane

    Stephane Dallongeville November 12, 2019 at 12:56 pm

    We updated both ROI Statistics and Workbook plugin which may fix some weird issues when number of ROI is > 65535 (wasn’t directly due to the size of XLSX file but more to the processing queue size). We also fixed a performance issue during workbook generation when ‘Color’ field was exported. Can you tell us if that fixed your problem ? Thanks.

    – Stephane

     
    deborakeller0@gmail.com November 13, 2019 at 11:25 am

    it again did not work… I get emtpy excel xlsx files – below the screenshot of the last image… snif
    deborakeller0@gmail.com November 13, 2019 at 4:44 pm

    one idea… could the available RAM limit the writing? ie I guess the intermediate results are stored somewhere, if there is not enough RAM dedicated to the storing for big images, could that play a role?

    I currenlty have 34.2Gb of RAM for icy and it does not seem to be using cache memory or disk…

    just a thought
    deborakeller0@gmail.com November 14, 2019 at 10:24 am

    I did another trial overnight (still running) with 3 images (193 time points, 900mB) and an output in TXT and it seems to have worked for the first time ever…

    though if I try and touch the program it will freeze

    However, it took a looong time and I have no clue why the 2nd image took so long

    Image1:
    Active contour: 1h20 (5521 ROI)
    spot detector: 8 min (199247 detected scale 1 90%, 109151 filtered >4px)
    ROI inclusion analysis: 1h42 min

    image 2:
    active contour 8h49min (5617 ROI – 5061 filtered)
    spot detector 10 min (189 961 ROI detected scale 1 90%, 98529 filtered >4px)
    ROI inclusion analysis: 58 min

    image 3 is still running active contour….

Viewing 7 posts - 1 through 7 (of 7 total)

The forum ‘General Issues’ is closed to new topics and replies.