Published on

Resources for self-learning programming

Authors

TODO:

  • New section for developer news and media
  • Check that all URLs are working

Preamble:

This is a collection of some of my favorite resources, content, courses, and everything else that I used to for the past 5 years (and counting). I wasn't very consistent with updating it, so it's probably missing a lot but I'll try to fill in the gaps slowly.

Having taught myself this far, I've learned a lot. So much so that I have a separate article detailing what I've learned about learning.

There's a mix of content types throughout the document. You'll find one-pagers, cheatsheets, article (and series of articles), videos (and series of videos), MOOCs, books, textbooks, and other creative ways education is distributed online. There is a large range in terms of emphasis of attention; some resources are more cosmetically inclined, others are more dense and utility-heavy. But everything here, in my opinion, is well made; or at the very least made with integrity. I am deeply thankful to the creators and providers of such content.

'Amateur.' you say that as if it was a dirty word or something, but 'amateur' comes from the Latin word 'amare' which means love... To do something for the love of it. - Mozart in the Jungle

This article is broken down into 3 sections:

  1. When I knew nothing
  2. When I knew some things
  3. What I wanna know now

I knew nothing:

Where everyone starts at some point. I started when I was 18

Language agnostic resources

  • At first, I didn't know which language to pick. I didn't even understand how to pick one. Getting started with an open mind helped me narrow down on which I should pick. It ended up being Python, but I tried many different languages before I settled.
    • I really wish I had know about this when I started programming.
    • Learn x in y minutes is a collection of "tours" for a variety of programming stuffs.
    • This Information-dense course by Stanford is called "The Missing Semester of your CS education", and it truly lives up to its name.
    • If you like learning by analogy or comparison, have a look at "Rosetta Code", a wiki of hundreds of programming languages demonstrating hundreds of tasks.
    • There's also "Hyperpolyglot", a simple site with side by side comparison charts of different programming languages organized by various catagories. Keep in mind the language versions in the demos are very old (the site isn't maintained).
    • If you want a high level overview of programming languages as a concept itself, check out this series of short videos.
    • I literally worship Runestone Academy's library of courses. Each little course serves as a great touch up on topics you may have encountered before, or as a way to get your feet wet in something new, as I have with the Java for Python programmers coursette
    • Level Up has great courses with videos covering lots of different topics.

Language specific resources

Python

  • Socratica is a YouTube channel for learning science and Python; their Python course is probably one of the best in quality, time, and understandability on the internet.
  • Codeacademy, something of a gatekeeper for learning coding — Just don’t use this alone and call it a day.
  • The Real Python website is the internet’s best collection of in depth tutorials and lessons on literally anything to do with Python. I would tell you to read everything but it would probably take years to go through all the amazing content.
  • And if you’re going to learn Python, it’s mandatory that you know NumPy.
  • It helped to be aware of tips and tricks like this, this, and this. You should know what are, and how to use dunder methods, decorators, and design patterns.
  • The response to this thread on Stack Exchange outlines the progression that most Python programmers will experience.
  • It will make a difference in everything else you learn and do. Here’s a list of Python’s best kept secrets that spell the difference between boys and me

Javascript

Java

  • This is jumping ahead a bit but the Algorithms in Java textbook by Robert Sedgewick is great for learning the ins and out of Java as well as algorithms in CS.

Data structures and Algorithms

  • Udacity has an incredible course called intro to “data structures and algorithms”, perfect for getting the seasoned programmer started with learning the concepts necessary to nail a developer position interview.

Developer tools

  • Thorough knowledge and familiarity with advanced programming tools. The quality of your programs will be hugely advanced, and some tools require knowledge in these concepts.
    • There is this wonderfully written list of the top mistakes beginner programmers make. Through experience I found a lot of these mistakes to be true, and are often amplified when the programmer in question is self-taught. Calling yourself out on these mistakes will make you a better programmer.
    • Organize your developer life with Git.
      • Read the Git flow and Github flow branching models.
      • https://learngitbranching.js.org/ is a highly recommended resource that lets you play with git without worrying about breaking anything.
      • Here's a great summary/cheatsheet for git.
      • GitHub and GitLab are two very popular repository management services that each have their uses.
      • Get familiar with Issues, Pull Requests, Licenses, OAuth, and automation tools like Github Actions.
    • Learn to read (and write) pseudocode.
    • Start learning terminal-level dev tools.
    • Go from command line to Powershell, customize your terminal.
      • Don't be too loyal to any one OS, install WSL to learn Linux.
      • It can help to learn how to use terminal tools like tmux for even more productivity.
      • One of the best courses for this is Learn Linux the hard way.
      • Learn how to ssh into any computer to navigate and control devices without a GUI.
      • As recommended by my dear friend Ananya: Linux Journey is a cute little website that teaches the ways of Linux.
      • If you're more of an interactive learner, I recommend learnshell.org and linuxsurvival.com.
      • If you want to get down and dirty, scripting is an important know-how (Here's another excellent guide on Shell programming).

I knew some things:

At a certain point the questions I had stopped being about languages or frameworks. I started wondering how best to structure my codebase, maximize my productivity, and optimize my code.

Design patterns

  • Design Patterns have their criticisms, but are useful as they are ubiquitous. If you can a hand of the OG GoF textbook online, definitely give it a go, otherwise, settle for "Head first: Design Patterns", or "Dive into Design Patterns".
  • YouTube has a great wealth of design pattern resources, one of my favorites of which is a playlist by Christopher Okhravi.
  • This brief video by William Lin, a competitive programmer provides a great outline on how to get into programming.

Vim

  • It can help to start learning vim (NOT emacs 😉).
    • Start with vimtutor (on mac and linux type in vimtutor in your terminal, on Windows... tough luck).
    • Also have a peek at some of the many vim cheatsheets (there are many, I just linked the most generic one).
    • In the meantime, use something like Anki to rote memorize commands.
    • Anki is for learning words, the vim grammar is for writing poetry.
    • Indoctrinate yourself as a vim vimmer by watching the top two (1, 2) vim tutorials.
    • Use vimgenius to practice, or vim adventures if you're into that kinda thing.
    • Pick up some basic vimscript along the way, and learn about .vimrc.
    • Get destroyed by the pros over at vimgolf.

Content

  • I also started understanding programming jokes and appreciating the humor.
    • How to be a -10x Engineer.
    • Programmers are also human is not only funny, but also helps distill a lot of the esoteric beliefs and topics in each of his videos.
    • If you're on YouTube a lot, you gotta subscribe to fireship; humor, education, and news all in one channel.
      • Also check out his courses (they're paid and I haven't tried them, but I'd trust them because I love the channel so much).

Mathematics

  • This very recent book (as of 2020) will probably blow up pretty soon.
  • This book by Googler Jeremy Kun is a fresh take on Math for Programmers Dr. Jeremy Kun - A Programmer’s Introduction to Mathematics (2018).pdf.
  • Check out his well-known blog too: Math ∩ Programming
  • Linear Algebra is helpful to understand how the layers of a NN work.
    • Have a look at MIT’s Linear Algebra course (taught by the great Gilbert Strang), available on MIT OpenCourseware's website.
    • For the best examples of concepts you should learn in lin alg, the Cliffnotes documents has a thorough albeit unstructured collection of notes.
    • I also really like Derek Banas YT channel for his linear algebra series (but his vegan recipes are very interesting).
  • Data Science
    • Udacity pulls through again, with another solid data course you can take for free. “Intro to Data analysis” will teach you the ins and outs of data science with Python, making use of CSVs, pandas, matplotlib, and more.
    • The StatQuest YouTube channel (brought to you by Josh Starmer) is one of the best channels for learning data and statistics for beginners. His dry humor and terrible song-writting makes for an equally informative and entertaining binge watching session.
    • Once you understand the theory and concepts, it’s time to learn how to implement. The Data School YouTube channel is the go to channel for all things data, reviewing concepts, highlighting best practices, and providing professional tips.
    • Here’s a repo with a basic cheatsheet of Data Science inspired by this repo with a Probability cheatsheet.
    • Get a grasp of data visualization with this collection of infographics containing all possible graphs, charts, and diagrams.

A.I

Machine learning

  • These models are not Neural Networks. These are your linear regressions, Kernel methods like Support Vector machines, Decision Trees, etc. keep in mind that Machine Learning isn't about the model, it's more about the method. Roughly speaking, there are 14 types of machine learning, and as the field develops, there will be more.
    • Udacity’s intro to A.I and machine learning is perfect for these pre-deep-learning concepts. Many times you will find (especially in times of scarce data), that these algorithms do a better job than the more popular neural network architectures.
    • The last resource you should take a look at is this wonderful paper called “A few useful things to know about machine learning”. These are 12 of the key takeaways, tips, and tricks, of machine learning.
    • In addition to the 12 tips, this article written by Patrick Riley of Google Advanced Science, highlights the 3 pitfalls most commonly seen in his real world experience at the intersection of machine learning and science.
    • Yannic Kilcher’s Youtube channel has a measly 3k subscribers but he deserves far more for his break-down of the newest and best machine learning papers. He has various conversations with expert researchers and frequently attends top conference, documenting both in videos of varying length. We need more channels like his.
    • How do you know whether or not you know your stuff? Google Data scien(tist)ce (interview) questions. If you can answer each one without help and within an accceptable degree of accuracy, you know your stuff.
    • Straight from Stanford’s CS 229 Deep Learning course, this repo contains cheatsheets to cover all basic machine learning needs. The same guys also made one for deep learning specifically (in the next section)
    • Mathematicalmonk’s channel. The most consistent and thorough youtube series, covers the explanation down to the math. Watch them all. Trust me it’s worth it.
    • Short and sweet, The 100 page Machine Learning book is a great primer into ML

Deep Learning

  • Neural Networks. The forefront of A.I research and focus. 70% of your time will be spent with NNs
    • Udacity Intro to Deep Learning is an obvious starting point. The course is relatively short and concise, and has just enough projects and content to give you a base level understanding of Deep Learning.
    • The deep learning book, written by Ian Goodfellow and Yoshua Bengio, is all you need to get familiar with the fundamentals of deep learning. This book is especially good at coinciding the math with the concepts.
    • What do neural networks learn? Is a great video that ties together math and neural networks. The video is structured such that even if you don’t know the math, you’ll still understand the computational differences between linear regression to neural nets and everything in between.
    • deeplearning.net and its documentation is one of the old-but-gold sites for learning everything deep learning. Curated by the makers of Theano, you’ll learn everything from Restricted Boltzmann Machines to Monte Carlo sampling. Each tutorial comes complete with a high level look at the math and the code (implemented in Python and Theano).
    • Arxiv Insights YouTube channel keeps to its namesake by publishing high qualiy videos of explainations behind some of the more complex topics in Deep Learning and RL. With proper references, clear visualizations, and thorough analysis, it’s a required watch.
    • ML From scratch’s blog post is the most thorough and holistic blog post on neural network basics. You don’t need any Medium articles or half-assed blog posts after this; everything else is just noise.
    • God bless these Stanford kids who made a repo containing cheat sheets for deep learning basics. It comprises of notes collected in Stanford’s CS 230 Deep Learning course.
    • I don’t know who this guy is, but for the past while, he has consistently put up silent short videos (2 mins on average) on his YouTube channel depcting the use of various Python and PyTorch tips and tricks. I watch these videos when I want to be productive without doing work. It’s fun to interpret what he’s doing and why something makes sense. I think that’s why it’s so compelling (kind of like Primitive Technology).
    • Convolutional Neural Networks, easily the most revered deep learning paradigm since its inception. There are literally thousands of resources, ranging from generic to overcomplicated. And in the eye of the storm, we have this.
    • Applicable to Machine Learning in general as well, but this blog post by Andrej Kaparthy started as a Twitter thread and became the biblical guide for intermediate ML engineers trying to train their models.
    • Deep Drizzle repo is a collection of the very very academic/university courses (video lectures included). If you can sit through any given video on 2x speed and understand everything without question, you’re a legend

Production Tools

  • Knowledge of the A.I libraries and packages available to you (You will probably learn about the packages and libraries as you go). Knowledge of the other tools available to you, like Google Colab (and Cloud). AWS, and various others is important. Understanding which is better for what purpose is key.
    • Udacity’s Intro to Deep Learning with PyTorch is possibly the greatest deep learning course currently available for free online. Created by Facebook A.I, the course covers everything from the fundamental basics of Deep Learning, all the way to CNNs and RNNs, all with guided PyTorch tutorials.
    • Tensorflow has the benefit of being backed by the biggest company in the world, and hence it has the most detailed documentation and series of tutorials than any other production tool on the internet. You can’t go wrong with their proprietary tutorial series.
    • Deep lizard is a YouTube channel that puts out consistent, concise, and high quality content. They have series on neural networks in Javascript and Python, and utilizing libraries from Keras to Pytorch. They even have series on RL. They deserve far more subscribers.
    • Jeff Heaton is probably the most likable machine learning tutor on YouTube. His tutorials cover the basics of deep learning, particularly with Keras and TF, and ranges from AWS and Flask pipelines all the way to visualization and preprocessing. Subscribe, if not for his content, then for his mustache.
    • The entire Google Suite at your disposal, most of which is free, some of which are paid per use. Amazon Web Services is also a must have for large projects or deployment level tasks.
    • Learning to visualize multidimensional data is a machine learning must. This medium article does a fantastic job explaining and demonstrating data visualization using an open source dataset using only matplotlib and seaborn, from 1D to 6D.
    • At surface level, there’s not much difference in between Keras, PyTorch, and TF. It’s only when you become an intermediate user that they each come into their own. It’s stuff like this and this that makes PyTorch consistent, intuitive, and highly flexible for professional users.
    • If you're a total badass then have a look at JAX (and it's precursors Autograd and XLA)
    • DeepMind's family of libraries (Haiku, RLax) make TensorFlow a bit more approachable, and if it's good enough for DeepMind, it's more than good enough for me
    • Parameter optimization
      • So you've built your model and want to make sure that it's the best version it can be. Enter hyperparameter tuning. There are multiple common methods ranging from large scale (GA algos and Grid Search) to more refined and empirical (Bayesian opt and Random Search). There are also multiple frameworks that support their development. Ray Tune is one of the few that has good-enough documentation and is kept up to date.
      • Botorch and Ax are PyTorch extension libraries meant for parameter optimization.

Blogs and things to follow

  • The ML explained blog is by a Carnegie Mellon university student who curates some quality content about once a month, explaining some more niche areas of deep learning as well as tutorials on building things from scratch.
  • Distill is so well curated that some people actually consider it to be on par with mid to top tier research papers in terms of quality of content. Many people on the notoriously serious ML subreddit also approve (which you should definitely follow).
  • Paperspace Blog is another one of those gems hidden in the pile of noise on the internet. Created by the Paperspace company (who does cloud and A.I stuff), the blog gathers quality content, approved by the company itself, who pays it’s writters with GPU credit (pretty clever).
  • Remember to try and keep your ego and biases away from studying. This Stack Exchange dicussion is an example of why something as simple as citation count should be regarded with skepticism.
  • Here are 7 websites to help you keep up to date with the rapidly changing world of machine learning:

Repositories and Tools

What I wanna know

Every day I learn something new about programming, even if it's something small. These are the things I'm currently learning and want to learn one day.

  • I'm currently getting better at Web development using:
    • Typescript
    • React
    • NextJS
    • Next-Auth
    • Redux
    • Zod
    • shadcn/ui
    • Tailwind CSS
    • React-icons
    • Framer Motion
    • Algolia
    • tRPC
    • GraphQL
    • Sendgrid
    • ApolloGraphQL
    • Prisma
    • Postgres
    • Vercel
    • GCP
    • Mixpanel
    • Docker
  • I really want to learn:
    • Rust (WASM, Tauri)
    • Go
    • Svelte (SvelteKit)
    • Swift
    • Kotlin (Jetpack Compose)

Discuss on Twitter • View on GitHub