Real-Time Semantic Mapping of Tree Topology Using Deep Learning and Multi-Sensor Factor Graph

Physical and structural tree measurements are applied in forestry, precision agriculture and conservation for various reasons. Since measuring tree properties manually is tedious, measurements from only a small subset of trees present in a forest, agricultural land or survey site are often used. Utilizing robotics to autonomously estimate physical tree dimensions would speed up the measurement or data collection process and allow for a much larger set of trees to be used in studies. In turn, this would allow studies to make more generalizable inferences about areas with trees. To this end, this thesis focuses on developing a system that generates a semantic representation of the topology of a tree in real-time. The first part describes a simulation environment and a real-world sensor suite to develop and test the tree mapping pipeline proposed in this thesis. The second part presents details of the proposed tree mapping pipeline. Stage one of the mapping pipeline utilizes a deep learning network to detect woody and cylindrical portions of a tree like trunks and branches based on popular semantic segmentation networks. Stage two of the pipeline proposes an algorithm to separate the detected portions of a tree into individual trunk and branch segments. The third stage implements an optimization algorithm to represent each segment parametrically as a cylinder. The fourth stage formulates a multi-sensor factor graph to incrementally integrate and optimize the semantic tree map while also fusing two forms of odometry. Finally, results from all the stages of the tree mapping pipeline using simulation and real-world data are presented. With these implementations, this thesis provides an end-to-end system to estimate tree topology through semantic representations for forestry and precision agriculture applications.