DeepSTORM3D: Dense Three Dimensional Localization Microscopy and Point Spread Function Design by Deep Learning

Localization microscopy is an imaging technique in which the positions ofindividual nanoscale point emitters (e.g. fluorescent molecules) are determinedat high precision from their images. This is the key ingredient insingle/multiple-particle-tracking and several super-resolution microscopyapproaches. Localization in three-dimensions (3D) can be performed by modifyingthe image that a point-source creates on the camera, namely, the point-spreadfunction (PSF). The PSF is engineered using additional optical elements to varydistinctively with the depth of the point-source. However, localizing multipleadjacent emitters in 3D poses a significant algorithmic challenge, due to thelateral overlap of their PSFs. Here, we train a neural network to receive animage containing densely overlapping PSFs of multiple emitters over a largeaxial range and output a list of their 3D positions. Furthermore, we then usethe network to design the optimal PSF for the multi-emitter case. Wedemonstrate our approach numerically as well as experimentally by 3D STORMimaging of mitochondria, and volumetric imaging of dozens offluorescently-labeled telomeres occupying a mammalian nucleus in a singlesnapshot.