Cell & Tissue Image Analysis: Time-lapse Microscopy and Cell Tracking - Prof. Badrinath Ro, Study Guides, Projects, Research of Electrical and Electronics Engineering

Download Cell & Tissue Image Analysis: Time-lapse Microscopy and Cell Tracking - Prof. Badrinath Ro and more Study Guides, Projects, Research Electrical and Electronics Engineering in PDF only on Docsity! ECSE-6963, BMED 6961 Cell & Tissue Image Analysis Lecture #23: Time-lapse Microscopy (Movies) Badri Roysam Rensselaer Polytechnic Institute, Troy, New York 12180. Center for Sub-Surface Sensing & Imaging Systems POSTER TITLE (in big letters, designed to provide an instant summary of your project, and attract an audience) Abstract of your project (200 words) References & Acknowledgment s Conclusions and Future Directions /Potential ResultsMethods (use diagrams, graphs, and equations, minimal text) Overview of your method, and key ideas Recap: Lineage Trees 96 Bee 35 ————_f aaa 54 t She at Bla 49 4 ia 46 t tea 45 dha 2 44———_f tae 3 a———_________j ia 5 ! aD 8 x u—f ha 31 { 310 27— 2a 20—f aa vw iva — ib Ll L L L L L L L 1 L 1 138 140 142 144 146 148 150 152 154 156 158 Time Post Fertilizatian(Min) Power Attenuator PMT 1 PMT 2 PMT 3 PMT 4 Image Signal Pulsed fs Ti-Sapphire Laser Wavelength Control Power Control Sync Signal Specimen Objective Lens Near-IR Excitation Light Short pass dichroic Visible Emission Light Mirror 495nm 515nm 560nm Long-pass dichroics (typical wave length cutoffs) PMT Gain Controls Image Acquisition Control Computer Resonant Scanning Mirrors Piezo Z control 5-D Microscope System 1 2 3 4 Tissue Microenvironments are Complex & Dynamic • Processes happen over time & space – Cell migration & transport – Structural dynamics – Driven by local interactions & relationships, and long-range gradients • Need to Understand – Location, nature, timing, & sequencing of critical events – Cell-cell & cell-vessel Interactions – Collective behaviors – Relevant 3-D spatial & temporal coordinates for all critical events Red: vessels Green: P14 thymocytes Blue: wild-type thymocytes Yellow: Dendritic cells (data: Ellen Robey, UCB) • High inter- & intra-dataset complexity Challenges Dense network of components Multiple component types Morphological Diversity Multiple behavior types Topological changes • Data volume exceeds manual analysis ability – Spatial: 512×512×21 voxels – Spectral: 4 color channels – Temporal: 40 frames per movie – Analysis cycle: weeks, months… • Unavoidable imaging artifacts Challenges Non-uniform illumination Spectral Cross-talk Spectrum Separation: Observations (X,Y,Z): 107,114,1 Intensity in green channel: 79 (X,Y,Z): 107,114,1 Intensity in red channel: 241 (X,Y,Z): 199,228,1 Intensity in green channel: 249 (X,Y,Z): 199,228,1 Intensity in red channel: 113 Red Green • Yellow Fluorescent Protein (YFP) signal is higher in the Red channel • Green Fluorescent Protein (GFP) signal dominates in the Green channel 2-step Thresholding Criteria • Averaged Intensity Ratio • Foreground/Background Cutoff Simple-minded Spectral Separation: Thresholding Approach NRed (i, j, k) NGreen (i, j, k) Neighborhood in Red channel Neighborhood in Green channel Unmixing Output: 2-channel case 2-channel 2-dye data Output Unmixed Image thymocytesGFPGreen Dendritic cellsYFPRed ObjectDyeChannel Input Raw Image Red Green Unmixing Example #1 Unmixing Output: 4-channel case • Segmentation of Thymocytes • Segmentation as Clustering – Intuitive description: cluster together voxels that belong to the same thymocyte – Solution: Mean-shift Clustering [Comaniciu et al., 2002] – Strengths: • No assumption about cluster shape • No prior knowledge in the number of clusters • Capable to handle arbitrary feature spaces (coordinates, intensity, etc.) • Only ONE tunable parameter – mean-shift window size h Cell Segmentation: Clustering-based Approach Break the voxel set P into small clusters {Cs}, where Cs is a group of voxels belonging to the sth thymocyteGoal A set of cells {Cs}, each with a feature vector fs (centroid, volume, etc.)Output A set of thymocyte voxels P = {pi |(xi , yi , zi)}Input Cell Segmentation: Mean-shift Clustering Algorithm Summary 1. Place an initial mean shift window centered at each voxel pi . 2. Compute new center location of each mean shift window Ωi . 3. Compute mean shift vector mi . 4. Repeat 2 & 3 until mi converges. 5. Merge voxels into one cluster Cs if their convergence points are closer than bandwidth h. Animation Source: www.wisdom.weizmann.ac.il/~deniss/vision_spring04/files/mean_shift/mean_shift.ppt • Enumerate all legal association hypotheses – Nodes: Cit at t and Cjt+1 at t+1 – Edge: hypothesis linking Cit with Cjt+1 – Weight: associating likelihood P • Calculate associating likelihood – Feature vector: fit and fjt+1 – Distance measure: di,jt,t+1 = ||fit - fjt+1|| – Migration model: Gaussian ~ (μ, Σ) – Parameter estimation: (μ, Σ) Recap: Multiple Hypothesis Tracking (MHT) t t+1 Cit Cjt+1 1 … 2 i M … 1 2 j N N-1 … …P • Pruned hypothesis set – Typically 3 best hypotheses per cell H = {hs | Ps} • Resolve optimal set of hypotheses H* – Maximize the cost function X * = arg Max{Σ xs × Ps} – Output is expressed in terms of binary values xs = 1 if hs is accepted hs→ H* xs = 0 if hs is ignored – Solved by integer programming [Al-kofahi et al., 2006] Multiple Hypothesis Tracking (Cont.) Cell Tracking: Experimental Output Tracked and color-coded Thymocytes projected on 2-D Plane 3-D View of Tracked ThymocytesMigration Trajectory of Thymocytes Experimental Results • Plot of Cumulative Distribution Function (CDF) – Difference in thymocyte-DC interactions between normal (wild type) and genetically modified thymocytes (P14) Validation: Limitations of Conventional Methods • Expensive in terms of manpower – Manual comparison of machine-generated output with ground-truth – Exhaustive search for potential errors – Highly dependent on subjective human observers • Not proportional to the accuracy of automated analysis – Many image analysis procedures are nearly mature and sufficiently reliable (i.e., accuracy > 90%) – Only a limited number of errors occur! • Automated Outlier Detection – Highlight potential errors only – Accept all other outputs • Interactive GUI – Visualize automated analysis – Inspect potential outliers – Editing Tool • Split, merge, add, delete • Group-based operation • Performance Assessment – Number of outliers – Number of editing operations – Processing time Outlier-driven Edit-based validation Outlier-based Method for Detecting Segmentation Failures • Types of Segmentation Failures – Under-segmentation: closing objects merge into large cluster – Over-segmentation: split a valid object to many sub fragments – False Detection: invalid objects due to imaging noise • Outlier Indicators – Utilize the nature of outliers as a strong hint Under-segmentation, Poorly-segmented errorsConcavity Depth Potential OutliersIndicator Over-segmentationAdjacency Under-segmentation, Poorly-segmented errorsShape Under-segmentationSolidity Under-segmentation, Over-segmentation, False detectionVolume Detection of Segmentation Failures (Cont.) • Volume (vi) of Object Small Small Large vi small Small Small f (vi) #9, # 48Falsely detected # 49Over-segmented # 3Under-segmented OutputType • Shape Fitting error – Poorly-segmented object – Under-segmentation Detection of Segmentation Failures (Cont.) fd) Detecting Split Cells ee * 3.5 + x * 3h * % Distribution of adjacency 2.57 based measure *# 2 L ¥ # * 15} % * * if + * 0.5 7 Split cells 4 a P tle) rf ‘ 1 1 1 \ 1 1 2 3 4 5 6 7 Adjacency Measure of Thymocyte (d;) Summary • We have learned cell tracking algorithms – A natural application of the Hungarian algorithm – We enhanced it to work with cells (they divide, for example) • Measuring cell-cell interactions in complex microenvironments using tracking results • Outliers are a good way to detect errors – They only pick up egregious errors but miss subtle errors • Next Class: – Analyzing longer-term events – Measuring associations in space and time – Comparing events across movies Instructor Contact Information Badri Roysam Professor of Electrical, Computer, & Systems Engineering Office: JEC 7010 Rensselaer Polytechnic Institute 110, 8th Street, Troy, New York 12180 Phone: (518) 276-8067 Fax: (518) 276-8715 Email: roysam@ecse.rpi.edu Website: http://www.ecse.rpi.edu/~roysam Course website: http://www.ecse.rpi.edu/~roysam/CTIA Secretary: Laraine Michaelides, JEC 7012, (518) 276 –8525, michal@.rpi.edu Grader: Center for Sub-Surface Imaging & Sensing