If you’ve tried to go past the APIs like the ones OpenAI offers and learn how they work “under the hood” by trying to build your own neural network, you might find yourself hitting a wall when the material opens with equations like this:
How can you learn how neural networks — or more accurately, artificial neural networks — do what they do without a degree in math, computer science, or engineering?
There are a couple of ways:
Follow this blog. Over the next few months, I’ll cover this topic, complete with getting you up to speed on the required math. Of course, if you’re feeling impatient…
Read Tariq Rashid’s book, Make Your Own Neural Network. Written for people who aren’t math, computer science, or engineering experts, it first shows you the principles behind neural networks and then leaps from the theoretical to the practical by taking those principles and turning them into working Python code.
Along the way, both I (in this blog) and Tariq (in his book) will trick you into learning a little science, a little math, and a little Python programming. In the end, you’ll understand the diagram above!
One more thing: if you prefer your learning via video…
My third experiment for 2024 involves trying out the ideas from Noah Kagan’s new book, Million Dollar Weekend.
ℹ️ In case you’re wondering: my first experiment of 2024 was to turn my layoff experience into a series of articles; the second was to take a chance working with a pre-seed startup.
Why conduct such an experiment? For now, let’s just say that current circumstances make it necessary, and hey, if anyone can pull off this kind of thing, it would be me.
The general idea of Million Dollar Weekend is that you can start a lucrative business by doing the following:
Identify a problem that you can solve
Solve that problem in a way that is hard to resist and profitable
Test your solution at low (or no) cost by preselling it before you build it.
The prerequisite for the Million Dollar Weekend process is a certain amount of unmitigated gall. Time and again in the book, Kagan states that two things hold people back from starting businesses:
Fear of starting
Fear of asking
Kagan’s methodology is to start by trying out an idea, seeing if someone will pay for that idea, and then either refining that idea or coming up with a new one and repeating the cycle.
The methodology anticipates rejection, and in fact, it says that in selling your idea, you should aim for plenty of rejections. The idea is that if you’re getting rejected often, you’re asking often, and that’s what eventually leads to success.
I’ll write more as I continue with this experiment, but for now, if you’re curious, here are some resources I can point you to:
You might also find these interviews with Kagan interesting:
ℹ️ Also in case you were wondering: This is NOT a paid promo for the book — neither Noah Kagan nor his businesses have any idea who I am or how to deposit money into my bank account. I wish they did!
I already own one of the three (Artificial Intelligence with Python, Second Edition), but I’m not going to complain about getting two new-to-me books at 50 cents each!
The books are:
Artificial Intelligence with Python, Second Edition, by Alberto Artasanchez and Prateek Joshi. This book is a great introduction to artificial intelligence via TensorfFow and Python, and great companion for the other two books in this one-dollar set.
Machine Learning with PyTorch and Scikit-Learn, by Sebastian Raschka, Yuxi (Hayden) Liu, and Vahid Mirjalili. PyTorch offers both neural networks and tensors, Scikit-Learn provides a collection of machine learning algorithms, and this book walks you through both.
Transformers for Natural Language Processing, Second Edition, by Denis Rothman. The “T” in ChatGPT is “transformer,” and this book covers them extensively. I’m looking forward to the exercise where you use Hugging Face to pretrain a RoBERTa model from scratch.
Are you looking for an economical way to become an AI development expert? Spend a buck on these three books, get any computer made in the past dozen years with plenty of RAM (SODIMMS for old computers are pretty cheap these days), and do the exercises in the books. We’re early enough into the new AI age that if you do all these, you’ll be ahead of most aspiring AI developers out there.
(I’m not affiliated with Manning in any way, other than I own some Manning books and get their announcement emails, which is how I found out about this.)
If your curiosity about artificial intelligence goes beyond bookmarking those incessant “10 ChatGPT prompts you need to know” posts that are all over LinkedIn, you should set aside some time to read Douglas’ Hofstadter’sGödel, Escher, Bach: An Eternal Golden Braid and watch his new interview.
Gödel, Escher, Bach
I might never have read it, if not for Dr. David Alex Lamb’s software engineering course at Queen’s University, whose curriculum included reading a book from a predetermined list and writing a report on it. I’ll admit that I first rolled my eyes at having to write a book report, but then noticed that one of the books had both “Escher” and “Bach” in the title. I had no idea who “Gödel” was, but I figured they were in good company, so I signed up to write the report on the book I would later come to know as “GEB.”
I’ll write more about why I think the book is important later. In the meantime, you should just know that it:
Helped me get a better understanding of a lot of underlying principles of mathematics and its not-too-distant relative, computer science, especially the concepts of loops and recursion
Advanced my thinking about how art, science, math, and music are intertwined, and inspired one of my favorite sayings: “Music is math you can feel”
Gave me my favorite explanations of regular expressions and the halting problem
Taught me that even the deepest, densest subject matter can be explained with whimsy
Provided me with my first serious introduction to ideas in cognitive science and artificial intelligence
Yes, this is one of those books that many people buy, read a chapter or two, and then put on their bookshelf, never to touch it again. Do not make that mistake. This book will reward your patience and perseverance by either exposing you to some great ideas, or validate some concepts that you may have already internalized.
At the very least, if you want to understand “classical” AI — that is AI based on symbol manipulation instead of the connectionist, “algebra, calculus, and stats in a trench coat” model of modern AI — you should Gödel, Escher, Bach.
A new Hofstadter interview!
Posted a mere three days ago at the time of writing, the video above is a conversation between Douglas Hofstadter and Amy Jo Kim. It’s worth watching, not only for Hofstadter’s stories about how GEB came to be, but also for his take on current-era large language models and other generative AI as well as the fact that he’s being interviewed by game designer Amy Jo Kim. Among other things, Kim was a systems designer on the team that made the game Rock Band and worked on the in-game social systems for The Sims.
There’s a Humble Bundle deal going on for the next nine days at the time of writing: the Popular Programming Languages book bundle, featuring books by O’Reilly. There’s a “15 books for $25” version of the deal and a “10 books for $18” version.
If you’re still iffy about parting with that much money, you might be interested in the $1 deal, which gets you these five books:
