Robust blind spectral unmixing for fluorescence microscopy using unsupervised learning

Autoři: Tristan D. McRae aff001;  David Oleksyn aff003;  Jim Miller aff003;  Yu-Rong Gao aff001
Působiště autorů: Multiphoton Research Core Facility, Shared Resource Laboratories, University of Rochester Medical Center, Rochester, NY, United States of America aff001;  Department of Neuroscience, University of Rochester Medical Center, Rochester, NY, United States of America aff002;  Center for Vaccine Biology and Immunology and Department of Microbiology and Immunology, University of Rochester Medical Center, Rochester, NY, United States of America aff003
Vyšlo v časopise: PLoS ONE 14(12)
Kategorie: Research Article


Due to the overlapping emission spectra of fluorophores, fluorescence microscopy images often have bleed-through problems, leading to a false positive detection. This problem is almost unavoidable when the samples are labeled with three or more fluorophores, and the situation is complicated even further when imaged under a multiphoton microscope. Several methods have been developed and commonly used by biologists for fluorescence microscopy spectral unmixing, such as linear unmixing, non-negative matrix factorization, deconvolution, and principal component analysis. However, they either require pre-knowledge of emission spectra or restrict the number of fluorophores to be the same as detection channels, which highly limits the real-world applications of those spectral unmixing methods. In this paper, we developed a robust and flexible spectral unmixing method: Learning Unsupervised Means of Spectra (LUMoS), which uses an unsupervised machine learning clustering method to learn individual fluorophores’ spectral signatures from mixed images, and blindly separate channels without restrictions on the number of fluorophores that can be imaged. This method highly expands the hardware capability of two-photon microscopy to simultaneously image more fluorophores than is possible with instrumentation alone. Experimental and simulated results demonstrated the robustness of LUMoS in multi-channel separations of two-photon microscopy images. We also extended the application of this method to background/autofluorescence removal and colocalization analysis. Lastly, we integrated this tool into ImageJ to offer an easy to use spectral unmixing tool for fluorescence imaging. LUMoS allows us to gain a higher spectral resolution and obtain a cleaner image without the need to upgrade the imaging hardware capabilities.

Klíčová slova:

Antigen-presenting cells – Fluorescence – Fluorescence imaging – Fluorescence microscopy – Imaging techniques – Lasers – Yellow fluorescent protein – Emission spectra


1. Denk W, Strickler JH, Webb WW. Two-photon laser scanning fluorescence microscopy. Science. 1990;248: 73–76. doi: 10.1126/science.2321027 2321027

2. Svoboda K, Yasuda R. Principles of two-photon excitation microscopy and its applications to neuroscience. Neuron. 2006;50: 823–839. doi: 10.1016/j.neuron.2006.05.019 16772166

3. Gao Y-R, Drew PJ. Effects of voluntary locomotion and calcitonin gene-related peptide on the dynamics of single dural vessels in awake mice. J Neurosci. 2016;36: 2503–2516. doi: 10.1523/JNEUROSCI.3665-15.2016 26911696

4. Gao Y-R, Greene SE, Drew PJ. Mechanical restriction of intracortical vessel dilation by brain tissue sculpts the hemodynamic response. NeuroImage. 2015;115: 162–176. doi: 10.1016/j.neuroimage.2015.04.054 25953632

5. Benninger RKP, Piston DW. Two‐photon excitation microscopy for the study of living cells and tissues. Current Protocols in Cell Biology. 2013;59: 4.11.1–4.11.24. doi: 10.1002/0471143030.cb0411s59 23728746

6. Xu C, Webb WW. Measurement of two-photon excitation cross sections of molecular fluorophores with data from 690 to 1050 nm. J Opt Soc Am B. 1996;13: 481–491. doi: 10.1364/JOSAB.13.000481

7. Drobizhev M, Tillo S, Makarov NS, Hughes TE, Rebane A. Absolute two-photon absorption spectra and two-photon brightness of orange and red fluorescent proteins. J Phys Chem B. 2009;113: 855–859. doi: 10.1021/jp8087379 19127988

8. Bestvater F, Spiess E, Stobrawa G, Hacker M, Feurer T, Porwol T, et al. Two‐photon fluorescence absorption and emission spectra of dyes relevant for cell imaging. J Microsc. 2002;208: 108–115. doi: 10.1046/j.1365-2818.2002.01074.x 12423261

9. Dickinson ME, Bearman G, Tille S, Lansford R, Fraser SE. Multi-spectral imaging and linear unmixing add a whole new dimension to laser scanning fluorescence microscopy. BioTechniques. 2001;31: 1272–1278. doi: 10.2144/01316bt01 11768655

10. Lansford R, Bearman G, Fraser SE. Resolution of multiple green fluorescent protein color variants and dyes using two-photon microscopy and imaging spectroscopy. J Biomed Opt. 2001;6: 311–318. doi: 10.1117/1.1383780 11516321

11. Neher R, Neher E. Optimizing imaging parameters for the separation of multiple labels in a fluorescence image. J Microsc. 2004;213: 46–62. doi: 10.1111/j.1365-2818.2004.01262.x 14678512

12. Zimmermann T, Rietdorf J, Girod A, Georget V, Pepperkok R. Spectral imaging and linear un‐mixing enables improved FRET efficiency with a novel GFP2–YFP FRET pair. FEBS Letters. 2002;531: 245–249. doi: 10.1016/s0014-5793(02)03508-1 12417320

13. Ecker RC, de Martin R, Steiner GE, Schmid JA. Application of spectral imaging microscopy in cytomics and fluorescence resonance energy transfer (FRET) analysis. Cytometry A. 2004;59A: 172–181. doi: 10.1002/cyto.a.20053 15170596

14. Nadrigny F, Rivals I, Hirrlinger PG, Koulakoff A, Personnaz L, Vernet M, et al. Detecting fluorescent protein expression and co-localisation on single secretory vesicles with linear spectral unmixing. Eur Biophys J. 2006;35: 533–547. doi: 10.1007/s00249-005-0040-8 16568270

15. Wildanger D, Vicidomini G, Bückers J, Kastrup L, Hell SW. Simultaneous multi-lifetime multi-color STED imaging for colocalization analyses. Opt Express. 2011;19: 3130–3143. doi: 10.1364/OE.19.003130 21369135

16. Valm AM, Oldenbourg R, Borisy GG. Multiplexed spectral imaging of 120 different fluorescent Labels. PLoS ONE. 2016;11: e0158495. doi: 10.1371/journal.pone.0158495 27391327

17. Tsurui H, Nishimura H, Hattori S, Hirose S, Okumura K, Shirai T. Seven-color fluorescence imaging of tissue samples based on fourier spectroscopy and singular value decomposition. J Histochem Cytochem. 2000;48: 653–662. doi: 10.1177/002215540004800509 10769049

18. Zimmermann T, Marrison J, Hogg K, O’Toole P. Clearing up the signal: spectral imaging and linear unmixing in fluorescence microscopy. Methods Mol Biol. 2014. pp. 129–148. doi: 10.1007/978-1-60761-847-8_5 24052349

19. Neher RA, Mitkovski M, Kirchhoff F, Neher E, Theis FJ, Zeug A. Blind source separation techniques for the decomposition of multiply labeled fluorescence images. Biophysical Journal. 2009;96: 3791–3800. doi: 10.1016/j.bpj.2008.10.068 19413985

20. Pengo T, Muñoz-Barrutia A, Zudaire I, Ortiz-de-Solorzano C. Efficient blind spectral unmixing of fluorescently labeled samples using multi-layer non-negative matrix factorization. PLoS ONE. 2013;8: e78504. doi: 10.1371/journal.pone.0078504 24260120

21. Lee DD, Seung HS. Learning the parts of objects by non-negative matrix factorization. Nature. 1999;401: 788–791. doi: 10.1038/44565 10548103

22. Montcuquet A-S, Hervé L, Navarro F, Dinten J-M, Mars JI. In vivo fluorescence spectra unmixing and autofluorescence removal by sparse nonnegative matrix factorization. IEEE Trans Biomed Eng. 2011;58: 2554–2565. doi: 10.1109/TBME.2011.2159382 21672672

23. Huang S, Zhao Y, Qin B. Two-hierarchical nonnegative matrix factorization distinguishing the fluorescent targets from autofluorescence for fluorescence imaging. Biomed Eng Online. 2015;14: 1–19. doi: 10.1186/1475-925X-14-1

24. Qin B, Hu C, Huang S. Target/background classification regularized nonnegative matrix factorization for fluorescence unmixing. IEEE Trans Instrum Meas. 2016;65: 874–889. doi: 10.1109/TIM.2016.2516318

25. Ricard C, Debarbieux FC. Six-color intravital two-photon imaging of brain tumors and their dynamic microenvironment. Front Cell Neurosci. 2014;8: 57. doi: 10.3389/fncel.2014.00057 24605087

26. Rakhymzhan A, Leben R, Zimmermann H, Günther R, Mex P, Reismann D, et al. Synergistic strategy for multicolor two-photon microscopy: application to the analysis of germinal center reactions in vivo. Sci Rep. 2017;7: 7101. doi: 10.1038/s41598-017-07165-0 28769068

27. Hinton GE, Sejnowski TJ, Poggio TA. Unsupervised learning. 1st ed. Cambridge: MIT Press; 1999.

28. Kapoor A, Singhal A. A comparative study of K-Means, K-Means++ and Fuzzy C-Means clustering algorithms. 2017 3rd IEEE International Conference on Computational Intelligence & Communication Technology. 2017. pp. 1–6. doi: 10.1109/CIACT.2017.7977272

29. Ishidoshiro N, Yamaguchi Y, Noda S, Asano Y, Kondo T, Kawakami Y, et al. Geological mapping by combining spectral unmixing and cluster analysis for hyperspectral data. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 2016. pp. 431–435. doi: 10.5194/isprs-archives-XLI-B8-431-2016

30. Torrecilla E, Stramski D, Reynolds RA, Millán-Núñez E, Piera J. Cluster analysis of hyperspectral optical data for discriminating phytoplankton pigment assemblages in the open ocean. Remote Sensing of Environment. 2011;115: 2578–2593. doi: 10.1016/j.rse.2011.05.014

31. Bunting P, Lucas RM, Jones K, Bean AR. Characterisation and mapping of forest communities by clustering individual tree crowns. Remote Sensing of Environment. 2010;114: 2536–2547. doi: 10.1016/j.rse.2010.05.030

32. Coates A, Ng AY. Learning feature representations with k-means. In: Montavon G, Orr GB, Muller KR, editors. Neural Networks: Tricks of the Trade. Berlin, Heidelberg: Springer; 2012. pp. 561–580. doi: 10.1007/978-3-642-35289-8_30

33. Sladitschek HL, Neveu PA. MXS-Chaining: a highly efficient cloning platform for imaging and flow cytometry approaches in mammalian systems. PLoS ONE. 2015;10: e0124958. doi: 10.1371/journal.pone.0124958 25909630

34. Sanchez-Lockhart M, Graf B, Miller J. Signals and sequences that control CD28 localization to the central region of the immunological synapse. The Journal of Immunology. 2008;181: 7639–7648. doi: 10.4049/jimmunol.181.11.7639 19017952

35. Mohamad IB, Usman D. Standardization and its effects on K-means clustering algorithm. Research Journal of Applied Sciences, Engineering and Technology. 2013;6: 3299–3303. doi: 10.19026/rjaset.6.3638

36. Arthur D, Vassilvitskii S. k-means++: the advantages of careful seeding. Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms. 2007. pp. 1027–1035.

37. Zimmermann T. Spectral imaging and linear unmixing in light microscopy. Adv Biochem Eng Biotechnol. 2005;95: 245–265. doi: 10.1007/b102216 16080271

38. Keshava N, Mustard JF. Spectral unmixing. IEEE Signal Processing Magazine. 2002;19: 44–57. doi: 10.1109/79.974727

39. Ricard C, Arroyo ED, He CX, Portera-Cailliau C, Lepousez G, Canepari M, et al. Two-photon probes for in vivo multicolor microscopy of the structure and signals of brain cells. Brain Struct Funct. 2018;223: 3011–3043. doi: 10.1007/s00429-018-1678-1 29748872

40. Huang H, Zhang P, Qiu K, Huang J, Chen Y, Ji L, et al. Mitochondrial dynamics tracking with two-photon phosphorescent terpyridyl iridium(III) complexes. Sci Rep. 2016;6: 20887. doi: 10.1038/srep20887 26864567

41. Albota MA, Xu C, Webb WW. Two-photon fluorescence excitation cross sections of biomolecular probes from 690 to 960 nm. Appl Opt. 1998;37: 7352–7356. doi: 10.1364/ao.37.007352 18301569

42. Dunn KW, Kamocka MM, McDonald JH. A practical guide to evaluating colocalization in biological microscopy. Am J Physiol Cell Physiol. 2011;300: C723–C742. doi: 10.1152/ajpcell.00462.2010 21209361

43. Croce AC, Bottiroli G. Autofluorescence spectroscopy and imaging: a tool for biomedical research and diagnosis. European Journal of Histochemistry. 2014;58: 2461. doi: 10.4081/ejh.2014.2461 25578980

44. Monici M. Cell and tissue autofluorescence research and diagnostic applications. Biotechnology Annual Review. 2005;11: 227–256. doi: 10.1016/S1387-2656(05)11007-2 16216779

45. Walter J. Spectral Unmixing Plugins. 2004. Database: Available from:

46. Gammon ST, Leevy MW, Gross S, George GW, Piwnica-Worms D. Spectral unmixing of multicolored bioluminescence emitted from heterogeneous biological sources. Anal Chem. 2006;78: 1520–1527. doi: 10.1021/ac051999h 16503603

47. Rueden CT, Schindelin J, Hiner MC, DeZonia BE, Walter AE, Arena ET, et al. ImageJ2: ImageJ for the next generation of scientific image data. BMC Bioinformatics. 2017;18: 529. doi: 10.1186/s12859-017-1934-z 29187165

48. Schindelin J, Arganda-Carreras I, Frise E, Kaynig V, Longair M, Pietzsch T, et al. Fiji: an open-source platform for biological-image analysis. Nat Methods. 2012;9: 676–682. doi: 10.1038/nmeth.2019 22743772

49. Lakowicz JR. Fluorophores. Principles of Fluorescence Spectroscopy. 2nd ed. Boston, MA: Springer, Boston, MA; 1999. pp. 63–93.

50. Shaner NC, Steinbach PA, Tsien RY. A guide to choosing fluorescent proteins. Nat Methods. 2005;2: 905–909. doi: 10.1038/nmeth819 16299475

51. Luker KE, Pata P, Shemiakina II, Pereverzeva A, Stacer AC, Shcherbo DS, et al. Comparative study reveals better far-red fluorescent protein for whole body imaging. Sci Rep. 2015;5: 10332. doi: 10.1038/srep10332 26035795

52. Akemann W, Sasaki M, Mutoh H, Imamura T, Honkura N, Knöpfel T. Two-photon voltage imaging using a genetically encoded voltage indicator. Sci Rep. 2013;3: 2231. doi: 10.1038/srep02231 23868559

53. Mahou P, Zimmerley M, Loulier K, Matho KS, Labroille G, Morin X, et al. Multicolor two-photon tissue imaging by wavelength mixing. Nat Methods. 2012;9: 815–818. doi: 10.1038/nmeth.2098 22772730

54. Dunn KW, Sandoval RM, Kelly KJ, Dagher PC, Tanner GA, Atkinson SJ, et al. Functional studies of the kidney of living animals using multicolor two-photon microscopy. Am J Physiol Cell Physiol. 2002;283: C905–C916. doi: 10.1152/ajpcell.00159.2002 12176747

55. Zimmermann T, Rietdorf J, Pepperkok R. Spectral imaging and its applications in live cell microscopy. FEBS Letters. 2003;546: 87–92. doi: 10.1016/s0014-5793(03)00521-0 12829241

56. Gobinet C, Perrin E, Huez R. Application of non-negative matrix factorization to fluorescence spectroscopy. 2004 12th European Signal Processing Conference. 2004. pp. 1095–1098.

57. Pu H, He W, Zhang G, Zhang Bin, Liu F, Zhang Y, et al. Separating structures of different fluorophore concentrations by principal component analysis on multispectral excitation-resolved fluorescence tomography images. Biomed Opt Express. 2013;4: 1829–1845. doi: 10.1364/BOE.4.001829 24156047

58. Dao L, Lucotte B, Glancy B, Chang LC, Hsu LY, Balaban RS. Use of independent component analysis to improve signal‐to‐noise ratio in multi‐probe fluorescence microscopy. J Microsc. 2014;256: 133–144. doi: 10.1111/jmi.12167 25159193

59. Pu H, Zhang G, He W, Liu F, Guang H, Zhang Y, et al. Resolving fluorophores by unmixing multispectral fluorescence tomography with independent component analysis. Phys Med Biol. 2014;59: 5025–5042. doi: 10.1088/0031-9155/59/17/5025 25119190

60. Buehler A, Razansky D, Glatz J, Deliolanis NC, Ntziachristos V. Blind source unmixing in multi-spectral optoacoustic tomography. Opt Express. 2011;19: 3175–3184. doi: 10.1364/OE.19.003175 21369139

61. Kwong JD, Messinger DW, Middleton WD. Hyperspectral clustering and unmixing for studying the ecology of state formation and complex societies. SPIE Proc. 2009; 7457: p. 74570E. doi: 10.1117/12.826354

62. Xenaki SD, Koutroumbas KD, Rontogiannis AA. A novel adaptive possibilistic clustering algorithm. IEEE Trans Fuzzy Syst. 2016;24: 791–810. doi: 10.1109/TFUZZ.2015.2486806

63. Keshava N. A survey of spectral unmixing algorithms. Lincoln Lab J. 2003;14: 44–78.

64. Bateson A, Curtiss B. A method for manual endmember selection and spectral unmixing. Remote Sensing of Environment. 1996;55: 229–243.

65. Palsson B, Sigurdsson J, Sveinsson JR, Ulfarsson MO. Hyperspectral unmixing using a neural network autoencoder. IEEE Access. 2018;6: 25646–25656. doi: 10.1109/ACCESS.2018.2818280

66. Zhao W, Liu Q, Lv Y, Qin B. Texture variation adaptive image denoising with nonlocal PCA. IEEE Trans on Image Process. 2019;28: 5537–5551. doi: 10.1109/TIP.2019.2916976 31135359

67. Iordache M-D, Bioucas-Dias JM, Plaza A. Total variation spatial regularization for sparse hyperspectral unmixing. IEEE Trans Geosci Remote Sensing. 2012;50: 4484–4502. doi: 10.1109/TGRS.2012.2191590

68. Megjhani M, de Sampaio PC, Carstens JL, Kalluri R, Roysam B. Morphologically constrained spectral unmixing by dictionary learning for multiplex fluorescence microscopy. Bioinformatics. 2017;33: 2182–2190. doi: 10.1093/bioinformatics/btx108 28334208

69. Bezdek JC. Pattern recognition with fuzzy objective function algorithms. 1st ed. Norwell, MA: Plenum Press; 1981.

70. Zare A, Gader P. Piece-wise convex spatial-spectral unmixing of hyperspectral imagery using possibilistic and fuzzy clustering. Taipei, Taiwan: IEEE; 2011. pp. 741–746. doi: 10.1109/FUZZY.2011.6007622

71. Khanmohammadi S, Adibeig N, Shanehbandy S. An improved overlapping k-means clustering method for medical applications. Expert Systems with Applications. 2017;67: 12–18. doi: 10.1016/j.eswa.2016.09.025

Článek vyšel v časopise


2019 Číslo 12
Nejčtenější tento týden