### Easy Robot Programming using Learning from Demonstration

Contracting dynamic system primitive (CDSP): An algorithm to learn the dynamics of human arm motions from human subject data collected by using a Microsoft Kinect sensor. To capture the complexity of human arm motion a neural network (NN) is used to represent the dynamics. The human arm trajectories for reaching operation are modeled as a stable dynamic system with contracting behavior towards the goal location. Periodic human motions are also modeled under the same framework. To take into consideration the contracting nature of the dynamic system, the NN parameters are learned subject to the contraction analysis constraints. An optimization problem is formulated by relaxing the non-convex contraction constraints to linear matrix inequality (LMI) constraints. The CDSP is able to adapt to situations for which the demonstrations are not available, e.g., an obstacle placed in the path.

Relevant publications:

- P. K. Thota, H. Ravichandar, A. P. Dani, “Learning and Synchronization of Movement Primitives for Bimanual Manipulation Tasks”, IEEE Conference on Decision and Control, 2016.
- H. Ravichandar, A. P. Dani, “Learning Contracting Nonlinear Dynamics from Human Demonstrations for Robot Motion Planning”, ASME Dynamics, Systems and Control Conference 2015 – Best Robotics Student Paper Award
- H. Ravichandar, Pavan kumar Thota, A. P. Dani, “Learning Periodic Motions from Human Demonstrations using Transverse Contraction Analysis”, IEEE American Control Conference, 2016.

### Human Intention Inference and Task Sequence Prediction for Human-Robot Collaboration

Adaptive Neural Intention Estimator (ANIE): An algorithm to infer the intent of a human operator’s arm movements based on the observations from a Microsoft Kinect sensor. Intentions are modeled as the goal locations of reaching motions in the 3-dimensional (3D) space. Human intention inference is a critical step towards realizing safe human-robot collaboration. Human arm’s nonlinear motion dynamics are modeled using an unknown nonlinear function with intentions represented as parameters. The unknown model is learned by using a neural network (NN). Based on the learned model, an approximate expectation-maximization (E-M) algorithm is developed to infer human intentions. Furthermore, an identifier-based online model-learning algorithm is developed to adapt to the variations in the arm motion dynamics, the motion trajectory, the goal locations, and the initial conditions of different human subjects.

Relevant publications:

- H. Ravichandar, A. Kumar, A. P. Dani, K. R. Pattipati, “Learning and Predicting Sequential Tasks using Recurrent Neural Networks and Multiple Model Filtering”, AAAI Symposium on Shared Autonomy in Research and Practice, 2016, pp: 331-337.
- H. Ravichandar, A. P. Dani, “Human Intention Inference using E-M Algorithm with Online Learning”, IEEE Transactions on Automation Science and Engineering, 2016, DOI: 10.1109/TASE.2016.2624279.
- H. Ravichandar, A. P. Dani, “Human Intention Inference using Artificial Neural Network-based E-M Algorithm”, IEEE/RSJ International Conference on Intelligent Robots and Systems, 2015.
- H. Ravichandar, A. P. Dani, “Human Intention Inference using Expectation-Maximization Algorithm with Online Model Learning” in “Human Modeling: System-level Investigation into Human Mechanisms for Assistive Technologies”, Edited by Jun Ueda and Yuichi Kurita, Elsevier, 2016.
- H. Ravichandar, A. Kumar, A. P. Dani, “Bayesian Human Intention Inference Through Multiple Model Filtering with Gaze-based Priors”, IEEE International Conference on Information Fusion, 2016.
- H. Ravichandar, A. P. Dani, ‘Human Intention Inference using Interacting Multiple Model Filtering’, IEEE International Conference on Multisensor Fusion and Information Integration for Intelligent Systems, 2015.

### GPS-denied Visual-Inertial Navigation

A vision-based localization and mapping algorithm was developed for an Unmanned Aerial Vehicle (UAV) which can operate in a riverine environment. Our algorithm estimates the 3D positions of point features along a river and the pose of the UAV. By detecting features surrounding a river and the corresponding reflections on the water’s surface, we can exploit multiple view geometry to enhance the observability of the estimation system. We use a robot-centric mapping framework to further improve the observability of the estimation system while reducing the computational burden. We analyze the performance of the proposed algorithm with numerical simulations and demonstrate its effectiveness through experiments. Our experimental platform is equipped with a lightweight monocular camera, an Inertial Measurement Unit (IMU), a magnetometer, an altimeter, and an on-board computer.

Vision Based Monocular Dense-SLAM(Simultaneous Localization and Mapping): There are two types of monocular SLAM: feature-based and dense monocular SLAM. The Dense-SLAM uses the intensity of the image frame for tracking and mapping, other than using the features such as the corners or edges. Compared with the feature-based monocular SLAM, the dense SLAM has the merits of providing more information about the geometry of the surroundings. Therefore, the Dense-SLAM could provide more information which is valuable for robotics.

Relevant publications:

- D. Chwa, A.P. Dani, and W. E. Dixon, “Range and Motion Estimation of Moving Objects using a Monocular Camera”, IEEE Transactions on Control Systems Technology, DOI: 10.1109/TCST.2015.2508001, 2015.
- J. Yang, A. P. Dani, S.-J. Chung, S. Hutchinson, “Vision-based Localization and Robot-centric Mapping in Riverine Environments”, Journal of Field Robotics, DOI: 10.1002/rob.21606, 2015.
- A.P. Dani, N. Fischer, and W. E. Dixon, “Single Camera Structure and Motion Estimation”, IEEE Transactions on Automatic Control, vol 57, No. 1, pp. 241-246, 2012.
- A.P. Dani, N. Fischer, Z. Kan, and W. E. Dixon, “Globally Exponentially Convergent Robust Observer for Vision-based Range Estimation”, Mechatronics, Special Issue on Visual Servoing, Vol. 22, No. 4, pp. 381-389, 2012.
- A. P. Dani, G. Panahandeh, S.-J. Chung, S. Hutchinson, “Image-moments for higher-level features based navigation”, IEEE/RSJ International Conference on Intelligent Robots & Systems, 2013, pp. 602-609.
- J. Yang, A. P. Dani, S.-J. Chung, S. Hutchinson, “Observer Design via Hybrid Contraction Analysis for UAS Navigation in Riverine Environments”, AIAA Guidance, Navigation, and Control (GNC), 2013.
- A. P. Dani, G. Panahandeh, S.-J. Chung, S. Hutchinson, “Image-moments for higher-level features based navigation”, IEEE/RSJ International Conference on Intelligent Robots & Systems, 2013, pp. 602-609.
- J. Yang, A. P. Dani, S.-J. Chung, S. Hutchinson, “Observer Design via Hybrid Contraction Analysis for UAS Navigation in Riverine Environments”, AIAA Guidance, Navigation, and Control (GNC), 2013.

**Image-based Tracking and Fusion**

User-guided Image-based Tracking: We are exploring image-based tracking algorithms based on different similarity measures and sensor fusion. An image tracking algorithm is developed that uses mutual information (MI) criteria for template matching and gyroscope information to predict rotation between two camera images. The tracking algorithm can also take an user input for template selection and update. The tracking algorithm uses Hu moments that are invariant to 2D rotation, translation and scaling to validate the tracker. Homography is used to represent template warping parameters. The algorithm is aided with gyroscope measurements to estimate the camera motion information which helps to improve the initial guess of the warping condition. Template selection using an user’s input is based on properties of the target, such as it’s location in the frame. The user driven strategy makes the tracker capable of tracking different objects of interest and might reduce the computational burden for template localization. The tracking algorithm presented in this paper shows significant improvements over recently developed gyro-aided Kanade-Lucas-Tomasi (KLT) tracker and the MI-only tracker and tracking in the case of multi-modal images. Recently, we have developed tracking algorithm based on Earth Mover’s distance as a similarity measure, called as iterative Earth Mover’s Distance tracker (iEMD-tracker). The iEMD tracker is shown to be robust to significant illumination changes, and partial occlusions.

Relevant publications:

- G. Yao, M. Williams, A. P. Dani, “Gyro-aided Visual Tracking Using Iterative Earth Mover’s Distance”, IEEE International Conference on Information Fusion, 2016. Best Student Paper Award – 2nd runner up.
- H. Ravichandar, A. P. Dani, “Gyro-aided Image-Based Tracking using Mutual Information Optimization and User Inputs”, IEEE International Conference on Systems, Man and Cybernetics, 2014.
- A. P. Dani, M. McCourt, J. W. Curtis, S. Mehta, “Information Fusion in Human-Robot Collaboration using Neural Network Representation”, IEEE Systems, Man, Cybernetics Conference, 2014, pp.2114-2120.

**Building Cooling Load Forecasting**

Relevant publications:

- P. K. Thota, A. P. Dani, P. B. Luh and S. Gupta, “Cooling Load Forecasting for Chiller Plants Using Similar Day based Wavelet Neural Networks”, IEEE International Conference on Complex Systems Engineering, 2015.

**Gait Estimation in Walking Restoration**

In this research estimation and control Algorithms for hybrid Exoskeleton-like Systems to aid walking are being explored. So far we have developed gait estimation algorithm using a nonlinear stochastic estimator based on state-dependent coefficient (SDC)-parametrization of nonlinear systems. For the estimator, the parameters of an electrically stimulated musculoskeletal model are estimated using approximate expectation maximization (E-M) algorithm. Inertial Measurement Units are used to estimate lower limb angles and complete walking gait is reconstructed using sensors attached to lower limbs. We are aimed at building a software to reconstruct the walking gait which can be used to make decisions on when to apply FES voltage, etc. This work is a joint collaboration with Prof. Nitin Sharma of University of Pittsburgh.

### System Identification

Identification of Electrically Stimulated Musculoskeletal Model: A system identification algorithm for a musculoskeletal system using an approximate expectation maximization (E-M) is presented. Effective control design for neuroprosthesis applications necessitates a well defined muscle model. A dynamic model of the lower leg with a fixed ankle is considered. The unknown parameters of the model are estimated using an approximate E-M algorithm based on knee angle measurements collected from an able-bodied subject during stimulated knee extension. The parameters estimated from the data are compared to reference values obtained by conducting experiments that separate the parameters in the dynamics from one another.

Relevant publications:

- H. Ravichandar, A. P. Dani, J. Khadijah-Hajdu, N. Kirsch, Q. Zhong, N. Sharma, “Expectation Maximization Method to Identify an Electrically Stimulated Musculoskeletal Model”, ASME Dynamics, Systems and Control Conference, 2015.
- A. P. Dani, N. Sharma, “A Discrete-time Nonlinear Estimator for an Orthosis-aided Gait”, ASME Dynamics Systems and Control Conference, 2014, paper no. DSCC2014-6161, pp. V001T04A003.
- N. Sharma, A. P. Dani, “Nonlinear estimation of gait kinematics during functional electric stimulation and orthosis-based walking”, American Controls Conference, 2014, pp. 4778-4783.