CUSEC 2010 Keynote: Matt Knox – “On Weakness”

This is the first of a series of notes that I took while attending CUSEC, the Canadian University Software Engineering Conference, which took place last week in Montreal. CUSEC is the biggest conference held by and for university students interested in software development. True to the Canadian techies punching well above their weight class (a great tradition started by Alexander Graham Bell), CUSEC manages to pull in big-name and up-and-coming speakers who’ve given talks that have outshined those I’ve seen an thousand-dollar-plus conferences.

The first keynote was given by Matt Knox, who has probably distributed more Scheme runtimes than anyone else in the world (and this is a larger number than you might think), which he did in the name of putting adware on millions of machines. He’s since come to his senses and seems quite contrite.

His presentation, On Weakness, is about his life on the Dark Side and the lessons he gleaned from it. It’s based on his talk, Crimes Against Humanity, Writ Small, which he gave at FutureRuby last year, but it was good to see it again, and its message is probably even more valuable to students. My notes (which I polished for comprehensibility) and photos from his session appear below:

An Evil Job

• How many of you are:
  • Technical, as opposed to business or arts students?
  • Engineering students?
  • Programmers?
  • Evil?
• That’s what this talk is about
• One way to describe one of my former jobs is doing “Windows hijinks with Scheme”
• During my time with that job, I released many scheme runtimes
• Aaron Swartz – I think it was at a Y Combinator startup camp – said this of me: "He uses Scheme for evil!"
• It was more than just Scheme – I was writing stuff that had alternately “hard” (statically-typed languages) and “soft” (dynamically-typed languages) layers
• I was in the adware business, which is like walking into a big monkey knife fight…
• …except I was using a death ray! (Scheme == death ray, C == knife)
• I started with good intentions, in the business of building spam filters
• Business wasn’t so hit, and I ran out of money
• My job search failed, but luckily, a job went looking for me
• I was so pleased with being found that I  forgot to talk salary
• I showed up for the interview and at the end, was invited to work for them
• I did terribly when it came time to discuss what I would be paid
  • I didn’t research the New York City job market and cost of living
  • I asked for $40K
  • When I saw the look of shock of the guy’s face, I thought that I had asked for too much
  • Start reducing what I asked for; luckily he stopped me
• We want you to come in an analyze our distribution chain, they said
• It turned out to be an adware company:
  • Bought people’s “digital tchochkes” or mini-apps, such as screensavers
  • They had realized that there’s no lower bound for how cheesy something can be and still be a big seller on the internet
  • They took these mini-apps and gave them away online for free, bundled with software that gives you "special offers" from time to time
• Some of these bundled apps turned out to be worms
  • So the company had me write software to remove any worms from a system and added them to the bundle
  • So now we were bundling my anti-malware along with their adware
  • I felt like "an assassin working for the mob, but killing terrorists". The mob were bad, but the terrorists were worse
  • "Awesome! I can probably keep up with Norton…it’ll be great!"
  • And for a while, the best way to eradicate worms your system was to install their adware with my anti-malware bundled with it
• Low-level coding is dangerously seductive
  • In the beginning, it’s "like getting kicked in the face over and over again by buffer overruns"
  • But then it becomes fascinating
• I wanted to do it in Scheme, but that would require embedding a Scheme interpreter
  • Such an interpreter would have to fit into a single TCP/IP packet (about 64K)
  • Scheme is great. For any superlative — “best performance”, “smallest app”, and so on – there are usually two contenders: some other language, and Scheme.
  • I managed to squeeze a Scheme interpreter down to 19K
• My success with killing the worms led to a new request: In addition to your all this malware on other machines, why not eliminate all the competitor’s adware?
  • Now I felt like “an assassin for the mob, killing other mobsters”. Not as noble.
• Then the next request came: How about keeping our software from being killed…by anything? (including Norton)
  • The only way to uninstall the adware was to use the uninstaller, which came with it
  • I initially viewed this as "a really interesting technical problem"
• All this was made possible by a couple of Windows quirks…
  • CreateRemoteThread
    • This basically says to a process: "Hey, process! Execute this code as part of yourself, and you’ll think it’s awesome."
    • This lets you have code executing even though a process isn’t running
    • You don’t even need threads – you can hook interrupts
  • Scheduler
    • You can have a process tell the scheduler that it needs to do a do-over — "I’m not done yet, I need more time", and the scheduler will grant that time
    • You can tell even Windows that a process is so important that if it fails, it needs to protect the user by presenting a blue screen
• Windows is interesting from a purely archaeological perspective
  • Consider that all strings in Windows are 16-bit unicode, which means that nulls can be embedded in strings
  • But C strings, which is what’s used in the underlying DOS, are null-terminated and therefore can’t contain nulls
  • Interesting effects when moving null-containing strings between these layers

What Drives People to Take Up Evil Jobs?

• Aftermath of my working at the adware company:
  • Company got sued for $190 billion (by Elliot Spitzer!)
  • I was the first employee at the company — everyone else was a contractor
• I left the company with these questions:
  • "Whut happen?"
  • "Is this who I am?"
• Some jobs pay lots of money, but it’s hard to transition out of them
• Will I be stuck in adware for the rest of my life?
• There are some historical precedents:
  • Albert Speer
    • A promising architect who liked soaring buildings
    • He hooked up with rising politicians with the same aesthetic sense, one of whom was Hitler
    • He started with creating buildings, but then became the Nazis’ chief logistics guy
    • Later, a leader of the U.S. Air Force said that had he been aware of Speer’s involvement as the Nazi’s chief logistics guy, he would’ve dedicated an entire wing of the Air Force exclusively to killing him
    • It’s been suggested that Speer prolonged the war by a year or two by running the German forces more efficiently
  • Manhattan Project staff
• But I didn’t want anecdotes…I wanted science!
  • There’s a scientific study of otherwise good people doing evil things: the Milgram Experiment
    • How many people would go all the way?
    • 1% of the population is psychotic – it was hypothesized that the number of people who’d go all the way would be similar
    • Instead, 70% did
    • Results replicatable with people from all walks of life
    • Women, it turned out, “went evil” in a slightly greater proportion than the men
    • "Most human evil lives here"
• Read The Black Book of Communism
• For a more mundane example of blind obedience to authority leading to evil, see "The strip search McDonald’s prank call"
  • In the prank, the prankster calls a McDonald’s, gets an employee on the line and says “I’m a police officer. We have reason to believe that there is a thief in your restaurant and we need you to take them into the back and hold them until we arrive.”
  • They provide a description vague enough so that someone in the restaurant will match it
  • Once coralled in the back, the prankster starts giving orders to torture and/or humiliate the customer, and many employees have complied
• So what does this mean?
  • The human brain has a remote root exploit in 70% of the installed base
• "With or without religion, you would have good people doing good things and evil people doing evil things. But for good people to do evil things, that takes religion." — Steven Weinberg
  • Nope. Just authority.
• There is hope: people who were subjects of the Milgram experiments turned out to be better at resisting authoritative coercion

The Power of Communication

• Math: "There are only three reasonable numbers: 0, 1 and infinity"
• When Robert Andrews Millikan did his oil drop experiments to determine the charge on an electron, he initially got the value wrong by 30 – 40%
  • People who repeated the experiment or conducted similar experiments with results close to Millikan’s erroneous number published their results
  • People who did so but got the correct value – which did not match Millikan’s value – didn;t publish, worried that they’d done something wrong, since their numbers didn’t agree with the number published by the authority on the subject
• The world pre-blogs was so different from this world
  • Very first open source project: Oxford English Dictionary
    • Done via mail
  • Ever wondered where the term "flying off the handle" comes from?
    • It’s from sword-making – until they figured out the process of making swords as one-piece, with hand-friendly stuff wrapped around the base so you could hold them, swords often flew off their handles in battle
    • It took 900 years to evolve swords to one piece
• Not everything has been solved, but it’s easier today
• Rails is such a solution
  • It’s a series of incremental improvements
  • Can you out-Rails Rails?

This article also appears in Canadian Developer Connection.