feature/thomas-intensity-changes-b - #243
Conversation
…separate BeamCorrectionParameters class that can be more easily reused by spark pipelines
…xlog zarr data from Google cloud storage
…ient and BeamCorrectionSparkClient runs
…CorrectionType parameter with possible values: OVERWRITE_SOURCE, WRITE_TO_TARGET, or SKIP
…sWithDifferentMFOVs method to removeMatchPairsThatReferenceTilesOutsideThisCollection
…service since we generate that match data in an earlier intensity correction step. The MFOVAsTileParameters deriveSfovMatchData field determines whether matches are simply read or are generated during the pre-alignment process. In "small" tests using 5 Google nodes with two w61_s140_r00 z layers, the run with deriveSfovMatchData=false took 61 minutes and the run with deriveSfovMatchData=true took 79 minutes.
…bute name consistent
…ive. This makes it a little clearer that you are excluding cross MFOV match pair patching when you specify a negative weight (rather than leaving the weight unspecified).
…lated MFOV patching is not done
…hat wasted compute time (e.g. 15 minutes of a 90-minute run)
minnerbe
left a comment
minnerbe
left a comment
There was a problem hiding this comment.
Thanks for putting this together, @trautmane! I have skimmed the code and only dug into a few targeted spots. All looks good to me. As always, my review comments are mostly just suggestions.
|
|
||
| if (excludeSameMfovNeighbors && (pMfov.equals(MultiSemUtilities.getMagcMfovForTileId(qTileId)))) { | ||
| continue; | ||
| } else if (excludeDifferentMfovNeighbors && (! pMfov.equals(MultiSemUtilities.getMagcMfovForTileId(qTileId)))) { |
There was a problem hiding this comment.
For readability, consider declaring final boolean isSameMfov = pMfov.equals(MultiSemUtilities.getMagcMfovForTileId(qTileId)).
There was a problem hiding this comment.
I agree adding isSameMfov would improve readability, but because this spot in the code is heavily used for FIB-SEM data sets I prefer to save time (for the FIBSEM data sets) by not doing the multi-SEM string conversion and comparison unless the parameters indicate this is a multi-SEM stack.
There was a problem hiding this comment.
I added a comment to explain this in the code
Co-authored-by: Michael Innerberger <innerbergerm@hhmi.org>
…-b' into feature/thomas-intensity-changes-b
2 participants
This pull request involves changing tile identifiers to improve render order, adding beam intensity correction as a spark pipeline step, and persisting/reusing within-MFOV match data during beam intensity correction and MFOV-as-tile processing.