Foundational Books for aspiring Machine Learning Engineers: A Curated Guide¶

In this blog post, I share eight books that I have found the most valuable to build the required skill set in the field of Machine Learning Engineering.

Introduction¶

Background¶

Below, I provide a little background before laying out the book recommendations, so you can decide whether these books are a good fit for the type of problems you wish to work on.

At the time of writing this blog post, which is December 2021, I had completed 4 years of university education in Statistics, earning both a Bachelor's and a Master's degree. My experience includes two data science internships and three years working full-time as a data scientist/machine learning engineer in three cities: London, Los Angeles, and Tokyo.

Please bear in mind that the book recommendations are based on the knowledge I sought after completing my Statistics degree. This means the recommendations are biased by my prior knowledge and the areas where I lacked knowledge, such as computer science. However, I did include one book from my university data mining course, which I still remember as absolutely amazing.

Specifically, when entering the job market, I was interested in building production software and solving business problems using machine learning and statistics.

Below is the knowledge and skill set to be gained from reading the books I recommend:

Software engineering
- Python programming
- Designing software systems
- Fundamental understanding of computers and information technology
- Algorithms and data structures
Machine learning
- Understanding and implementing machine learning algorithms
- Theoretical understanding of statistical learning and data mining
- Practical applications of machine learning in real-world scenarios
Business acumen in data science and machine learning
- Strategic decision-making
- Problem-solving ability to leverage data science for business solutions

Tip

Each book is tagged with a specific skillset, so that you can pick and choose the ones relevant to you.

Book List¶

The Hidden Language of Computer Hardware and Software¶

Link

Skillset to be gained:

Fundamental understanding of computers and information technology.

Why read this book?

This book, I think, is the most eye-opening book for anyone without a prior understanding of computers and with a curiosity to understand maybe the main technology of this world.

Every few pages you read will give you that aha moment 💡 where you feel like you just understand this world a lot better.

The book empowers you with the sufficient knowledge to intuitively understand popular tools such as Numpy.

You will understand what it means that Numpy efficiently allocates memory and what it means that it benefits from vectorisation and by utilising CPU’s available operations better.
Unless you understand how a computer memory and CPU works, you cannot understand the concepts of memory allocation, vectorisation or memory buses.

Achieve enlightenment 🌟 in some certain way, since the knowledge you gain will allow you to understand how a computer fundamentally works.

This allows you to become a better software engineer, since you can reason better about the problems you are solving and thus solve those problems better and faster.

This book will give you answers to the following questions:

How do 0’s and 1’s in the computer become letters we understand? How do letters we understand become 0’s and 1’s in the computer?
How does data flow through a cable or the air? How does electricity become 0 or 1?
How can a letter N be converted to a 0's and 1's?
How does a computer store that 0 or 1? What does it mean that something is 0 or 1 in a computer?
How does a computer add two numbers together such as 10 + 5? What about subtraction?
How does a computer run a code you write in Python or C++?
How do QR codes work?
How does the Braille writing system work?
What is CPU and random access memory and how to build them up from scratch using relays? And what is a relay?
Why do you lose all the data in your random access memory if you switch off the power on your computer?

This book is very well written, however, it requires deep focus and re-wiring of your brain as you read it. It might hurt a little .

Designing Data-Intensive Applications¶

Link

Skillset to be gained:

Designing software systems.

Why read this book?

Be able to design optimal software systems for your problem statement.

Be well-prepared for system design job interviews in machine learning engineering or software engineering in general.

This book will give you answers to the following questions:

What is a database? What is the difference between storing your data in a file system vs. using a database? Why use a database?
What is the difference between various database types and how to reason about choosing an optimal one for your problem statement?
How to design your system to optimise for fast queries of data?
How to store huge amounts of data whilst having fast queries?
What is distributed storage and how does it work? What are the trade-offs using such storage compared to a single-node storage?
How does an SQL query work? How is an SQL query converted to a logic that a computer uses to find the relevant data?
Why is a database query often a lot faster than finding your data using Python/bash and iterating through the files?
How to make queries of your data faster? What are the common methods? Gain understanding of indexing, partitioning, redundancy, replication, normalisation, hashing, etc.
Why is it usually easier to query relational vs non-relational database?
What is the optimal data model for your problem statement?

I recommend reading this book after you had read Hidden Language of Computer Hardware and Software, since this book requires prior knowledge of how computers work.

It’d be quite useful if you had some prior exposure to building software, dealing with storage of data and designing data models before to reading this, since you will have more context.

Cracking the Coding Interview¶

Link

Skillset to be gained:

Algorithms and data structures.

Why read this book?

Make it a habit and a second nature to take into account computational and space complexity when writing software.

This will help you write scalable code and identify bottlenecks.

Gain understanding of the available arsenal you have when writing software and be able to use it efficiently.

In other words, be able to choose right data structures for your problem statement and suitable type of algorithms.

Be able to pass job interviews since it’s very popular to test for the knowledge of data structures and algorithms.

Using this book for job interviews preparation:

I started reading this book to prepare for job interviews at Google in Tokyo in mid-2020 and I solved around 3-8 coding challenges per day which took me around 2-4 hours. I did this for about 15-20 days. So, I solved around 100 questions in total.

The number of questions I was able to complete per day varies, since sometimes I would get stuck for an hour on a question.

When solving the questions, I did not put myself under time pressure, which was maybe a mistake looking back, but well it was hard enough to motivate myself to spend like 50 hours on this book.

In the end, I did not get the job. However, practice does make perfect and if I did this practice for 40 days instead of 20, I think I would have passed those interviews. Google recommended to complete between 100-200 questions.

Solutions under pressure just did not come naturally enough for me in those interviews. The answers start coming naturally right after the interview is finished 😂.

I recommend reading this book after you had read Hidden Language of Computer Hardware and Software, since you will understand the data structures and algorithms better and more intuitively.

Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow¶

Link

Skillset to be gained:

Machine learning algorithms.

Why read this book?

Covers all the machine learning concepts with the perfect level of detail to give you the confidence in your understanding.

You will be able to explain and reason about all the methods and how they work, as well as be able to choose the optimal tool for solving your problem.

Since I had learnt about most of the ML techniques at university, I mainly focused on reading the deep learning section of this book.

An Introduction to Statistical Learning: with Applications in R¶

Link

Skillset to be gained:

Theoretical understanding on statistical learning and data mining.

Why read this book?

Provides the theoretical background and the fundamental building blocks behind machine learning.

Honestly, I think this the type of knowledge many people in ML who come from computer science and other non-statistical backgrounds miss 🤫.

There will be some overlap of topics with the other recommended book Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow. However, the actual content is not overlapping I believe, since each book is focused on different aspects.

This book uses R, but you don’t have to run the code examples and just focus on the knowledge.

Analytical Skills for AI & Data Science¶

Link

Skillset to be gained:

Business acumen.

Why read this book?

Builds up your business acumen and gives you the skillset to be able to look at real-life business problems and come up with data science/ML solutions to tackle them whilst taking into account the costs and benefits of various approaches.

This book helps you to build a problem solving toolkit to take complex problems, break them down and frame them in a systematic way.

Some case studies this book covers:

How would a store select optimal location to open a new branch?
Say you can predict that a customer is likely to churn, what is the best action to take? Is it worth offering a discount or let the customer go? If a discount, how much discount?
How much would it cost to clean all the windows in Mexico City?

This book does not seem to be that popular in the mainstream, but I myself felt so lucky to have found this book soon after it came out and felt that it will become a huge hit since its pretty amazing.

Also, I found this book soon after I failed an interview process in the last stage with Google for not having a strongly enough developed business acumen. I wish I knew about the book prior to the interviews since this is the perfect book to build up your business acumen for data science.

Fluent Python¶

Link

Skillset to be gained:

Expertise in Python.

Why read this book?

Get to know Python inside out and be able to use it efficiently to write software.

I loved this book since it’s an all-in-one deal about Python. Once reading it, I had the knowledge to use Python with confidence and joy.

High Performance Python¶

Link

Skillset to be gained:

Deeper understanding of Python and it's performance.

Fundamental understanding of computers.

Why read this book?

Learn about profiling your code to detect bottlenecks in terms of memory or CPU and then optimize.
Learn how Python data structures work in detail and how to use them appropriately for your problem.
Gain a deep understanding on Numpy.
Get a deeper understanding of computer memory and CPU and how it relates to performance of your code.

I recommend reading this book after you had read the books Hidden Language of Computer Hardware and Software and Fluent Python, since you need some fundamental understanding of computers and Python to be able to understand this book.

How to search for good technical books to read¶

Committing to reading a technical book should be taken seriously 🧐, since it takes a very long time to read compared to novels and is often a hard work to actually understand the content.

Myself, I have three ways to search for good technical books to read:

Refer to the Hackernews Top Books of All Time chart.
Use Google with a Hackernews keyword like “hackernews books for software design” and read the discussions.
Ask friends who are knowledgable of a certain topic to recommend me the best books on that topic.