Course of the tract Data Science of MSIAM and MOSIG master of Universite Grenoble Alpes.



  • Course
    • Introduction to convex optimization: concepts in convex analysis (duality, proximal operators), how to identify potential difficulties in optimization problems. Illustrations in supervised learning (classification and regression problems) and in operation research (decomposition methods
    • Algorithms in convex optimization (gradient, proximal gradient, conditional gradient, ADMM)
    • Stochastic gradient and Incremental algorithms (SGD, SAGA, SVRG)
    • Introduction to distributed computation (architectures for computation, map-reduce scheme, MPI, Spark) + practical work
    • Distributed optimisation algorithms, stochastic algorithms, asynchronous methods
  • Tutorials
    • Incremental Algorithms
    • Introduction to Spark
    • Sparse logistic regression in high dimension
    • Application to a recommendation system


The final grade will be a convex combination of the grade on the report on the practical sessions and the grade of the presentation of a recent research article.
  • Report on the practical sessions.
    We would like you to write a report on the two sessions "sparse regression" and "matrix completion", by groups of 1, 2, or 3 students. The format of the report is free; we expect between 2 and 7 pages, presenting an overview of your work with a focus on a (or several) specific aspect(s).
    We do not expect you to give all the answers, question by question. We do not expect either you to cover all the material of the two sessions. On the other hand, you can work out other developments.
    You can emphasize any aspect of your work, depending on your personal interests and skills, for instance:
    • implementation and numerical tests (further developments, more experiments,...)
    • applications in learning or statistics (interpretation of results, other models, other datasets...)
    • theoretical or mathematical questions (convergence proof of algorithms, convergence rates, advanced versions, theoretical analysis of special case...)
    Enjoy :) The report has to be sent before Friday, Dec 22 at
    The quality of presentation and of the analysis will obviously matters for the grade.

  • Presentation of research articles.
    We would like you to present an article by groups of 1, 2, or 3 students. The article has to be chosen in a list that we will given Monday, Dec. 18. The list contains various articles around the topics of the course: some are more theoretical, some are more algorithmic, others deal with applications. The presentation will be short: 8 mins + around 5 mins of questions. In this short time, you can present an overview of the article or put an emphasis on a specific aspect that you find interesting. The slides (in pdf) will be projected from our machine (if you want to present an implementation or a script run, you should prepare slides on it). The presentation will be in January. The presentation slides should be sent the day before the defense.
Note finally that we will have a last session on machine in January for you to ask question on the course, practical labs, or articles. Good work !