Atanas Mirchev

Atanas Mirchev

Research scientist at Volkswagen AG, TUM alumnus

Welcome!

My PhD dissertation is available on mediatum.ub.tum.de, before you read it please note the following errata and clarifications.

Technical points

  • The abstract says VI smoothing works reliably for indoor trajectories and I mentioned it works consistently for up to 4 m/s on p. 57. In hindsight these are not accurate as blanket statements – the 2021 paper has results for such data, but that is not exhaustive confirmation. For instance, there was an experiment in 2022 where I applied the smoother off-the-shelf to 25 Blackbird trajectories (I think 2 overlapped with the 2021 ones). From the plots I found, it seems it worked on 17 out of the box, but on 3 it built up drift, and the rest failed with no visuals (unclear why). The point is the method can work for different trajectories, but there is no guarantee it cannot fail, and the thesis does not analyze its robustness. As direct SLAM its generalization across scenarios is sensitive to tuning, and I did not properly analyze how resilient it is to data and model stochasticity (I vaguely recall rare stochastic failures due to VI are possible, unless my memory deceives me). The general points above apply to the PRISM filter too.

    Also, all inference results are on research benchmarks, and I assume both inference methods and their published hyperparameters can be brittle for many real-world tasks, given that a large error at any timestep can ruin a whole run. For example, if memory serves, running PRISM using the QCar to obtain a map for section 5.6 took multiple failed trials and careful motion to get a complete final run. Here acceleration control inputs were set to zero which is not ideal (assuming a constant-velocity transition at each timestep, no inertial data). I suspect a good dynamics model / IMU measurements and better RGB-D quality may help in some of these cases, but I can’t be sure – not all issues can be solved through data, the devil is in the details.

  • Also note that both the VI smoother and the PRISM filter provide uncertainty conceptually, but they are approximate solutions. Beyond map infogain exploration, this uncertainty might well not be sufficient for belief-state POMDP control (also see 6.1). Beyond the theoretical compromises, for example as per 5.2.2 (please note fig. 5.4 is only an example, the smoother is mean-field diagonal Gaussian), 5.3.2, 6.1, and the paper derivations, I also feel uncertainty calibration is still unresolved. PRISM’s appendix only scratched the surface of analyzing state calibration (no analysis for the map), there I had to correct the inferred Gaussian scales of both the smoother and the filter by global constants (if memory serves, I think the smoother ones especially were too low, likely because of initialization and convergence). To an extent this depends on the generative uncertainty, which was also not calibrated rigorously (e.g. global scale hyperparameters were mainly tuned for inference success). Personally, I think there is still a substantial gap between my ideal vision and the results, in terms of section 1.5.2 and certain desirable aspects that go beyond chapter 3 (e.g. calibration, moving away from independent factorizations, multi-modal posteriors, etc.) – I see the two inferences as early proofs of concepts, the remaining issues would require more work.

  • On p. 58, the last two sentences of 5.2 about uses of the VI smoother are hypothetical.

  • Fig. 1 in the PRISM paper is for illustration, IIRC the uncertainty contours were scaled up (i.e. don’t correspond to a standard deviation).

  • In the related work I listed traditional grid-based RBPFs (e.g. [1], [2]) next to landmark-based methods which was not the right place, particularly because the following paragraphs were a comment on preprocessed observations. Also, I should have pointed out that classical grid-based RBPFs already had an integration of a dense map in an SSM and raw range observations, even if lower-dimensional than modern dense RGB-D SLAM.

    The RBPF method in [3] also used a 3D ray-tracing sonar emission, with a small number of rays for speed. Its setup is different (underwater coarse octree maps, tunnel movement, water-pressure altitude sensors, 54 sonar beams shaped as pencils, etc.) from the RGB-D one I focused on, but it is similar in trying to scale grid-based probabilistic SLAM to 3D. I wish I had included it for that aspect, IIRC I found it after submitting the thesis and meant to add it for a while, but around the defense I focused on a few other SSM-like RGB-D methods I wanted to cite and decided not to add it in the end, and I think that was a mistake.

    More generally, notions from the grid-based RBPF line of work can be traced all the way back to [4] and also appear in the dissertation (e.g. grids, occupancy, stepping along rays and raytracing, closed-form map updates alike those in PRISM, etc.). And of course RBPFs provide complete posterior distributions, just like the other classical-era SLAMs in the related work (e.g. EKFs), which the dissertation strives to do as well, but with different inference algorithms (e.g. PRISM has a similar Bayes-filter recursion, but works differently). Combined, these analogies span the spectrum of thesis aspects (see sec. 3.3), the difference in the thesis is in the underlying model assumptions & inference methods, designed to fit both RGB-D sensors and free 6-DoF movement. Please note the similarity of such methods and interpret the contributions in that context.

  • To add to the above: if I am not mistaken [5] was the first grid-based RBPF to be applied to real-world 2D data. You can also see [6] for a modern attempt to mix a particle filter (not Rao-Blackwellized) with a 3D TSDF map and a non-ray-tracing implicit emission – this one is for RGB-D data like my work, it is another late find I decided not to add while I could and I should have. It is related in general and also to PRISM, with a different inference algorithm and a glimpse at PF robustness. From what I understand it seems it used a zero-mean Gaussian motion model without controls – in theory replacing that with a control-driven transition should be straightforward, which could then be used for predictive rollouts and then one could see the whole as an alternative probabilistic dense RGB-D SSM-and-filter combo for model-based control. This is how it connects to my perspective.

  • In summary, classical-era EKF & RBPF SLAMs from the 2000s already had a unity of probabilistic SSM & spatial assumptions (I hope this is clear from the related work on robotics, as I noted there they are POMDP-compliant). And as you can see from the above, grid RBPFs already had an SSM with a dense map, and at least one paper (e.g. [3]) was about scaling that to 3D. The aim of my thesis was to present a similar unified perspective for modern RGB-D dense SLAM and highlight the link to control, hopefully adding to the perspectives that exist already (see my related work, the RBPFs and [6] above, etc.).

    I think the last point should be clear if you read end-to-end, and if you see isolated mentions of “dense SLAM” and it is unclear what I meant from the context, note that the go-to prototypical example I had in mind was generating images with thousands of pixels (e.g. as in contemporary direct RGB-D or NERF methods). I do not recall how well I separated modern high-resolution (colored) 3D lidars in my mind in these cases, but in hindsight I think it is cleaner to treat them as separate, because the thesis has solely RGB-D results and I am also more aware of RGB(-D) than lidar trends.

  • On p. 68, line 3 I discuss a dynamics term in eq. 5.19 as new. When I wrote this line I was not aware of some overlap with prior works (e.g. [7], [8], [9], [10], [11]) which fuse dense visual frame-to-model tracking with external pose estimates (e.g. from wheel encoders, robot joint measurements or IMU data). This is the same idea as eq. 5.19, I believe with some minor technical differences (e.g. map and rendering details) and possibly a slight difference in perspective (controlled transition vs. motion prior from measurements).

  • On p. 74 I mention that there is some novelty in basing infogain exploration on VSSM-LM’s rendering and PRISM’s map filter (see Ziqing’s work on exploration). Here I want to point out [12] which uses 2D TSDF maps and is overall quite similar, particularly in its map updates (modulo some technicalities: 2D vs. 3D, rendering, map update & uncertainty truncation, candidate generation). Note that such exploration is very traditional, if you are interested some early works are [13], [14], [15] and you can find a few more modern 3D ones in the related work.

  • Apropos: classical grid-based RBPFs also harmonize with such exploration (see [15]).

After the defense, I added [7], [8], [9] last-minute as part of a more general discussion in the related work section on p. 45 (apropos, “forward kinematics and inertial/odometry loss terms” would be more accurate there), but I did not point out the above. Please take care if you cite.

On a different note: appendix C extends the background in Part I, the sources on p. 9 apply here too. On the whole, the background theory should be covered by those textbooks and the inline references, though I remember consulting a few more sources on these topics I did not cite and I spotted one I did not cite well, so I want to highlight them here.

Modulo only notation, the importance sampling derivation in C.29-C.31 and the reparam. trick formulation in C.26 are the same as in [16] by Maximilian Sölch and the POMDP transition definition in equation 1.33 is the same as in [17] by Baris Kayalibay, I remember liking their presentation and replicating them in my work (by the way, \(1/\mathcal{Z}\) in C.30 should be \(\mathcal{Z}\)). I also presented my ancestral sampling example in 1.1 very similarly to [16]. I also recall looking to [17] for ideas on how to introduce the control background, which resulted in presentation similarities in section 1.5 (e.g. the listing of MDP components, the fact I mentioned the Bellman optimality operator, etc.). And I think [16] influenced my writing and specifically my probabilistic background because I read it in the early writing stages (e.g. I see similarities in sections 1.1-1.2.1, or my comment on SSM independences in 1.4, there is probably more). In hindsight I should have handled all the above better and left a few citations. Appendix C.2 is a brief recap of Gauss-Newton and MAP – here I cited “Factor Graphs for Robot Perception” by Dellaert and Kaess at the end of the section when I mentioned sparsity, but what probably did not come across is that the book is a holistic reference, note that you can get what I had to say in C.2 from it. I also remember looking up the generalized Gauss-Newton equations in C.2 in this lecture by Fred Roosta, that would be equations C.15-C.18. For the theory on cameras, rotations and Lie groups I used these lectures by Daniel Cremers (e.g. IIRC definitions 2.16-2.18 were based on them, with presentation influenced by [18]; also, almost all equations in C.8,C.9 and the Lie-group parts in 2.1 are simultaneously both from there and the micro-Lie-theory tutorial I referenced, the sources overlap).

References

  1. [1]Murphy, Kevin P.: Bayesian Map Learning in Dynamic Environments. In: Advances in Neural Information Processing Systems 12, NIPS Conference, Denver, Colorado, USA, November 29 - December 4, 1999, 1999, pp. 1015–1021
  2. [2]Grisetti, Giorgio ; Stachniss, Cyrill ; Burgard, Wolfram: Improving Grid-based SLAM with Rao-Blackwellized Particle Filters by Adaptive Proposals and Selective Resampling. In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation, ICRA 2005, April 18-22, 2005, Barcelona, Spain : IEEE, 2005, pp. 2432–2437
  3. [3]Fairfield, Nathaniel ; Kantor, George ; Wettergreen, David: Real-Time SLAM with Octree Evidence Grids for Exploration in Underwater Tunnels. In: J. Field Robotics vols. 24 (2007), Nr. 1-2, pp. 03–21
  4. [4]Moravec, Hans P. ; Elfes, Alberto: High resolution maps from wide angle sonar. In: Proceedings of the 1985 IEEE International Conference on Robotics and Automation, St. Louis, Missouri, USA, March 25-28, 1985 : IEEE, 1985, pp. 116–121
  5. [5]Hähnel, Dirk ; Burgard, Wolfram ; Fox, Dieter ; Thrun, Sebastian: An efficient fastSLAM algorithm for generating maps of large-scale cyclic environments from raw laser range measurements. In: 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems, Las Vegas, Nevada, USA, October 27 - November 1, 2003 : IEEE, 2003, pp. 206–211
  6. [6]Höhner, Nils ; Hebborn, Anna Katharina ; Müller, Stefan: Particle Filter Based Tracking and Mapping. In: Bebis, G. ; Boyle, R. ; Parvin, B. ; Koracin, D. ; Turek, M. ; Ramalingam, S. ; Xu, K. ; Lin, S. ; et al. (eds.): Advances in Visual Computing. Cham : Springer International Publishing, 2018 — ISBN 978-3-030-03801-4, pp. 299–308
  7. [7]Wagner, René ; Frese, Udo ; Bäuml, Berthold: Graph SLAM with signed distance function maps on a humanoid robot. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2014, Chicago, IL, USA, September 14-18, 2014 : IEEE, 2014, pp. 2691–2698
  8. [8]Klingensmith, Matthew ; Sirinivasa, Siddartha S. ; Kaess, Michael: Articulated Robot Motion for Simultaneous Localization and Mapping (ARM-SLAM). In: IEEE Robotics Autom. Lett. vols. 1 (2016), Nr. 2, pp. 1156–1163
  9. [9]Scona, Raluca ; Nobili, Simona ; Petillot, Yvan R. ; Fallon, Maurice F.: Direct visual SLAM fusing proprioception for a humanoid robot. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2017, Vancouver, BC, Canada, September 24-28, 2017 : IEEE, 2017, pp. 1419–1426
  10. [10]Laidlow, Tristan ; Bloesch, Michael ; Li, Wenbin ; Leutenegger, Stefan: Dense RGB-D-inertial SLAM with map deformations. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2017, Vancouver, BC, Canada, September 24-28, 2017 : IEEE, 2017, pp. 6741–6748
  11. [11]Houseago, Charlie ; Bloesch, Michael ; Leutenegger, Stefan: KO-Fusion: Dense Visual SLAM with Tightly-Coupled Kinematic and Odometric Tracking. In: International Conference on Robotics and Automation, ICRA 2019, Montreal, QC, Canada, May 20-24, 2019 : IEEE, 2019, pp. 4054–4060
  12. [12]Saulnier, Kelsey ; Atanasov, Nikolay ; Pappas, George J. ; Kumar, Vijay: Information Theoretic Active Exploration in Signed Distance Fields. In: 2020 IEEE International Conference on Robotics and Automation, ICRA 2020, Paris, France, May 31 - August 31, 2020 : IEEE, 2020, pp. 4080–4085
  13. [13]Stachniss, Cyrill ; Burgard, Wolfram: Exploring Unknown Environments with Mobile Robots using Coverage Maps. In: Gottlob, G. ; Walsh, T. (eds.): IJCAI-03, Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, Acapulco, Mexico, August 9-15, 2003 : Morgan Kaufmann, 2003, pp. 1127–1134
  14. [14]Triebel, Rudolph ; Frank, Barbara ; Meyer, Jörg ; Burgard, Wolfram: First steps towards a robotic system for flexible volumetric mapping of indoor environments. In: IFAC Proceedings Volumes vols. 37 (2004), Nr. 8, pp. 651–656. — IFAC/EURON Symposium on Intelligent Autonomous Vehicles, Lisbon, Portugal, 5-7 July 2004
  15. [15]Stachniss, Cyrill: Robotic Mapping and Exploration, Springer Tracts in Advanced Robotics. vols. 55 : Springer, 2009 — ISBN 978-3-642-01096-5
  16. [16]Sölch, Maximilian: Uncovering dynamics: Learning and amortized inference for state-space models, Technical University of Munich, Germany, PhD thesis, 2021
  17. [17]Kayalibay, Baris: Control with Spatial World Models (Regelung mit Räumlichen Weltmodellen), TU Munich, Germany, PhD thesis, 2024
  18. [18]Engel, Jakob-Julian: Large-Scale Direct SLAM and 3D Reconstruction in Real-Time, Technische Universität München, PhD thesis, 2017