The study of most scientific fields now relies on an ever-increasing amount of data, due to instrumental and experimental progress in monitoring and manipulating complex systems made of many microscopic constituents. How can we make sense of such data, and use them to enhance our understanding of biological, physical, and chemical systems? Aimed at graduate students in physics, applied mathematics, and computational biology, the primary objective of this textbook is to introduce the concepts and methods necessary to answer this question at the intersection of probability theory, statistics, optimisation, statistical physics, inference, and machine learning. The second objective of this book is to provide practical applications for these methods, which will allow students to assimilate the underlying ideas and techniques. While readers of this textbook will need basic knowledge in programming (Python or an equivalent language), the main emphasis is not on mathematical rigour, but on the development of intuition and the deep connections with statistical physics.