Course Presentations
1. Adaptive Lambda Least Squares Temporal Difference Learning
This project is a part of the coursework on Reinforcement Learning at IISc, Bangalore. This is a team project that had two members including me. We tried understanding the work of Timothy et al. on Adaptive Lambda Least Squares TD Learning, a technical report at Arxiv. We also implemented the novel algorithm of this paper, which chooses the lambda parameter adaptively and balances Bias-Variance trade off in TD. The presentation on the same is attached for reference. We also gave an overall picture of TD learning in the presentation.
2. Semi Definite Programming Method for Integer Convex Quadratic Minimization
In this project, I studied the NP hard problem of minimizing convex quadratic function over integer lattice. I analyzed the paper "A semi definite programming method for integer convex quadratic minimization" by J. Park and S. Boyd, where a simple semi-definite programming (SDP) relaxation is carried out for obtaining nontrivial lower bound on optimal value of the problem and a randomized algorithm is carried out to find sub-optimal solution. The optimal values obtained by solving the problem of various sizes using the above formulation were listed by numerical simulations. Various methods to obtain the bounds on optimal values of the problem were also studied.
3. Importance Sampling
A short presentation on Importance Sampling.
4. Conjugate Gradient Descent
A short presentation on conjugate gradient descent.
5. Linear Estimation in Krein Spaces
In this presentation, I discussed the application of Krein-space projection in H_{\infty} problem. The presentation discussed the results in the following two papers:
“Linear estimation in Krein spaces. I. Theory” Hassibi, B.; Sayed, A.H.; Kailath, T. Automatic Control, IEEE Transactions on Volume: 41 1, Jan. 1996, Page(s): 18 -33
“Linear estimation in Krein spaces. II. Applications” Hassibi, B.; Sayed, A.H.; Kailath, T. Automatic Control, IEEE Transactions on Volume: 41 1, Jan. 1996, Page(s): 34 -49
This presentation was a part of the PhD course “Mathematical Methods in Signals, Systems and Control” that I credited at KTH.