In all my years, I’ve failed it only once. But I’m certain that actually experiencing that failure ensured that the lesson would “stick.”
I happened a few years back. I was being diligent and getting all my tax stuff ready to send to my accountant in early February, around the time when my then-employer was sending employees their primary tax document, the dreaded Form W-2. (For those of you outside the U.S., it’s the wage and tax document provided by your employer; for example, the Canadian equivalent is the “T4 Slip”.)
I was doing a search through my company inbox to find the download location for my W-2 information, having forgotten that it was available through Workday. One of the search results was one of those phishing email tests, disguised to look like an official email with a link to my tax info. Since I was reading the email as search results and not as email, I was not in my usual email security mindset, clicked the link in the email, and boom:
I got the usual “Your manager will be notified and you’ll have to undergo mandatory security re-education” message afterward. Surprisingly, my manager never brought it up, and I was never scheduled for the “Don’t do it again, dumbass” remedial course, but believe me: I learned my lesson that day.
Here it is — the video of my presentation, xz made EZ, which covers the security incident with the xz utils utility on Unix-y systems, which I gave at BSides Tampa 2024 on April 6th:
The details of the xz vulnerability were made public mere days before the BSides Tampa 2024 cybersecurity conference, and on a whim, I emailed the organizers and asked if I could do a lightning talk on the topic.
They quickly got back to me and let me know that they’d had a last-minute speaker cancellation and gave me a full slot in which to do my presentation.
The moral of the story? It never hurts to ask, and it can lead to opportunities!
What’s this xz thing, anyway?
Let me answer with this slide from my presentation:
xz is short for xz Utils, a compression utility that you’ll find in Unix-y operating systems, including:
Linux distributions
macOS
It’s usually used by Unix greybeards who generally use it in combination with tar.
What happened with xz?
xz was one of those open source projects that had a vulnerability best illustrated by this xkcd comic:
xz was like that project pointed out in the comic, except that the “random person” doing the maintaining was Lass Collin, a developer based in Finland, who was experiencing burnout. As a result, xz was languishing.
In what appeared to be a stroke of good fortune, a developer who went by the handle of “Jia Tan” on GitHub came to the rescue and started submitting patches to xz.
At about the same time, there were a number of complaints about xz’s lack of apparent maintenance. In hindsight, it looks like a clever two-pronged campaign:
A group of people loudly clamoring for someone else to take the reins of the xz project, and
A friendly developer who swoops in at the right time, making patches to the xz project…
…all while a burned-out Lasse Collin was facing a lot of stress.
On November 30, 2022, Lasse changed the email address for xz bug reports to an alias that redirected to both his email address as well as Jia Tan’s. At that point, Jia Tan, the apparently helpful developer who appeared at just the right time, was now an official co-maintainer.
Not long after, Lasse releases his last version of xz, and soon after Jia Tan, now the sole maintainer of the project, releases their own version.
With full control of the project, Jia Tan starts making changes — all the while, carefully disguising them — that create a “back door” within the xz application.
On any system that had Jia Tan’s tainted version of xz installed, an unauthorized user with the right private key could SSH into that system with root-level access. By becoming the maintainer of a trusted application used by many Linux versions, Jia Tan managed to create a vulnerability by what could have been one of the most devastating supply-chain attacks ever.
It’s back! The 11th edition of BSides Tampa, Tampa Bay’s community-led cybersercurity conference, happens Saturday, April 6th at Marshall Student Center at USF.
You’ll want to attend BSides if:
You work in cybersecurity, because your peers — some of whom you might not know — will be there.
You’re looking for a cybersecurity job. You’ll get to network with people in the field, and you’ll find the conference’s Career track helpful.
You’re curious about cybersecurity. What do cybersecurity people do? They test systems for vulnerabilities (go to the talks in the Offense / Red Team track), they protect systems from attackers (go to the talks in the Defense / Blue Team track), and they create processes to enhance security (go to the talks in the Governance track).
You’re into intelligence — human and artificial. There’s an AI / Defense track that covers these topics.
You want to learn. I can’t think of a BSides where I didn’t learn at least three important things.
You want to know what the Tampa security scene is like. Tampa has an underappreciated security scene, and you’ll get to see what it’s like at BSides Tampa!
BSides Tampa is sponsored by the Tampa Bay chapter of (ISC)², which is clever and mathematically-correct shorthand for “International Information System Security Certification Consortium”. (ISC)² is a non-profit specializing in training and certifying information security professionals.
BSides gets it name from “b-side,” the alternate side of a vinyl or cassette single, where the a-side has the primary content and the b-side is the bonus or additional content.
Here’s the origin story: When the 2009 Black Hat conference in Las Vegas received more presentation submissions than they could take on. There were many presenters whose talks weren’t accepted, but were still very good — there just wasn’t enough room for them at Balck Hat.
So they banded together and made their own parallel conference that ran in parallel to Black Hat — it’s from that event that we get BSides.
BSides conferences are community events, and unlike a lot of tech conferences, they’re inexpensive. BSides Tampa 2024 costs $45 to attend — the same price as last year — and that gets you:
Access to all conference tracks
Access to Discord server
Access to the exhibition area, villages, and sponsorship area
Last year’s CyberX Tampa Bay event was a big hit, and it was only natural that there’d be another one this year. Like the first one, this year’s event was packed.
The moment I walked into the venue, I saw so many people and had so many conversations that I never got the chance to take pictures until the start of the “welcome” session in the large room:
…after which we had the choice of two breakout sessions:
Chronicles of an Entry-level Cybersecurity Professional
The Wheel of Misfortune
I went to the Wheel of Misfortune, where audience members got the chance to answer cybersecurity questions for Google swag. Anyone in the audience could volunteer to come up to the front, spin the wheel of topics and answer a question based on that topic.
Hosts Jason Allen and Jonas Kelley were pretty relaxed about audience assistance. At one point, I yelled out the acronym for remebering the 7 layers of the OSI network model — “Please Do Not Take Sausage Pizza Away!” — and no one was penalized.
The room, where every seat and available spot to stand was occupied, was lively, with people enjoying themselves. The audience participation, aided by two engaging hosts, kept the room lively until the very end.
It was then time to recognize CyberX Tampa Bay’s 2023 honoree — someone nominated by attendees as being the person who made the biggest positive impact on Tampa Bay’s cybersecurity scene. This year’s honoree was Jeremy Rasmussen!
And to close the evening, there was the keynote panel on cybersecurity myths. It featured…
Photo by Kasandra Perez. Tap to view at full size.
I attended BSides St. Pete last Saturday, the second anniversary of this event, and it was nice to see that attendance had more than doubled. It’s nice to see the that the Tampa Bay cybersecurity community is active on both sides of “The Other Bay Area!”
BSides gets it name from “b-side,” the alternate side of a vinyl or cassette single, where the a-side has the primary content and the b-side is the bonus or additional content. In 2009, when the Black Hat conference in Las Vegas received way more presentation submissions than they could take on, the rejected presenters (who still had very could presentations; there just wasn’t enough capacity for them) banded together and made their own “b-side” conference that ran in parallel with Black Hat. From that event came BSides.
Opening keynote: Between Two Palms: A Session on Burnout
The day started at 9 with the opening keynote, which took place not only on the main stage, but between two palm plants, as promised in its title:
The keynote was a frank discussion moderated by John “Cochise” Buzin (one of my instructors at the UC Baseline cybersecurity course I took in the summer of 2020) and featured Chris Machowski (also one of the people behind the UC Baseline course) and Elvira Reyes.
While they stated quite clearly that they aren’t psychology professionals, they are very active in the cybersecurity field, and each of them knows something about burnout from personal experience.
Over their talk, they talked about what they identified as the five stages of burnout, starting with stage one, the honeymoon phase:
This stage is marked by the following:
Job satisfaction
Accepting responsibility
Sustained energy levels
Unbridled optimism
Commitment to the job
Compulsion to prove oneself
Free-flowing creativity
High productivity levels
Stage two is the onset of stress:
In this stage, you’ll experience:
CV symptoms
Inability to focus
Irritability
Reduced sleep quality
Lack of social interaction
Lower productivity
Anxiety
Avoidance of decision-making
Change in appetite
Headache
Neglect of personal needs
Fatigue
Then comes stage three — chronic stress:
Symptoms of this stage include:
Persistent tiredness
Procrastination
Resentfulness
Social withdrawal
Aggressive behavior
Apathy
Chronic exhaustion
Cynical attitude
Decreased sexual desire
Denial of problems
Feeling threatened
Feeling pressured
Alcohol/drug consumption
Next, stage 4, burnout:
Here’s what you’ll experience in this stage:
Obsession with problems
Pessimistic outlook
Physical symptoms
Self-doubt
Social isolation
Chronic headaches
Chronic GI problems
Neglect of personal needs
Escapist activities
Behavioral changes
And finally, stage 5 — habitual burnout:
And with this comes:
Chronic sadness
Chronic mental fatugue
Chronic physical fatigue
Depression
After this rather gloomy description of burnout’s stages came the things you can do to counter burnout:
They generally boil down to “take better care of yourself,” which is in agreement with what the Mayo Clinic says.
I thought their use of the iconography from the Fallout games for the topic of burnout was pretty clever.
Anonymous trooper
I passed by this fella on the way to the next session:
How to build a cybersecurity journey
I caught a bit of Ivan Marchany’s session, How to Build a Cybersecurity Journey, one of the presentations that covered how one gets into the business of cybersecurity.
Among other things, he covered building your own cybersecurity lab…
…and reminded the audience that as far as prospective employers and clients are concerned, you are your projects:
And equally important is the fact that if you don’t have some kind of online presence in this day and age, you effectively don’t exist to employers and clients:
This was a popular topic, and Ivan was playing to a standing-room-only audience:
Cyber risk management
I also caught the tail end of Dan Holland’s presentation, Complexity is the Enemy: How to start doing Cyber Risk Management. I’m pretty sure I arrived at one of the most important slides, the “risk as a product of probability and impact” slide:
I plan to share this slide on the Okta Slack’s “random” channel:
And here are the takeaways from Dan’s presentation:
A Urinal Story: Human Behavior & Security
Somehow, I managed to miss the “urinal story” part of Daniel Lopez’ and Ashwini Machlanski’s presentation on helping firm up the human element in cybersecurity. They covered key parts of managing people through the use of behavioral science and little tricks like “nudges” to get people to be more security-compliant.
This slide summarizes their key takeaways quite well:
Ashwini and Daniel handed out my favorite stickers from the conference:
My one tragic mistake
In wandering the halls and checking out what was happening in other rooms, I failed to catch Stacey Oneal’s Getting into Cybersecurity presentation, which was on my list. I owe her one — I promise I’ll catch you at your next presentation, Stacey!
Super Grouper hadn’t opened by the time I got to the trucks, so I got an Elvis Burger from 1 Up. It’s been a while since I last had a peanut butter-and-bacon burger, and I enjoyed mine. I know it sounds weird, but it’s worth trying!
Lunch keynote: Becoming a Proactive Defender
While having lunch, I caught most of Christopher Peacock’s presentation, Becoming a Proactive Defender:
I’m going to steal his line, “The best teacher is the adversary; the adversary always gets a vote.”
IAM Security and So Can You: An Intro to Identity Access Management and How to Beat It to a Pulp
I’ve been told that there was a presenter at BSides Tampa that was a bit of dick and overdid it with his bad-mouthing Okta while I wasn’t in the room, so while this talk featured a different presenter, you’d better bet your ass that I was going to be at this one.
But Jarred “Raydar” Pemberton was a lot more reasonable than the other guy. He got an intro from Cochise, who not only mentored him, but convinced him that he should give this presentation. That was a good call; in matters of cybersecurity, if Cochise suggests you do something, it’s generally a good idea to do it.
“Does SSO scare red teamers?” Jarrad asked. “Yes,” he plied to his own question, saying that it’s the kind of thing he shied away from.
Jarrad told us about what he does for a living. It’s always fascinating to see how people who use the stuff we make work with it:
Take note of that last point: in addition to the HR staff or outside HR consultants like “The Bobs,” another person that might be at your termination meeting is someone whose job is to close your work accounts.
If you’re ever unfortunate enough to be a guest at a layoff meeting, you may encounter “The Bobs” (a term from the film Office Space). Find out more about them here.
I’m actually on the Auth0 side of Okta, which provides a service for customer logins, versus the Okta side of Okta, which handles SSO (single sign-on) for the workforce. My experience with the Okta service is mostly as a user: I use it to log into systems at work:
Yup, that’s an Okta slide! Jarrad’s take on Okta:
“One that I work a lot with and do like quite a bit”
“Super easy to use”
“Simple to get brought up to speed”
“It’s what I would recommend to an org if they can afford it”
(Note to self: Send Jarrad some swag.)
SSO, in addition to letting a workforce since into various work systems with a single set of credentials, has other uses, including certain HR-related tasks:
Monitoring access and, by virtue of knowing who’s logging into what, see who’s really coming into the office and who’s merely pretending to do so
Easily hitting the “off” button for an employee when necessary
Jarrad then went into the different types of SSO, starting with cookie sharing. It’s typically used with internally-developed applications, such as home-grown HR and payroll applications at less mature organizations that haven’t graduated to SaaS application, and if those applications have a common parent domain (that is, if they live on an URL of the form *.your-domain-here.your-tld-here. He recommends against it, as it’s pretty much broken.
He then talked about SAML — Security Assertion Markup Language — an open-standard, XML-based framework for authentication and authorization between two entities without a password.
Want to know more about OAuth? Check out my teammate Matt Raible’s article, What the Heck is OAuth?
Most of his talk was focused on the standard that also happens to be my livelihood: OAuth or Open Authorization, the open standard for access delegation, which is often used to grant websites or applications access to user information without giving them their login credentials.
He also quickly mentioned Kerberos, which is for authenticating requests among trusted hosts on an untrusted network:
Here’s some good advice from all you pentesters. Be sure to follow them, especially that last one:
It’s not the early 2000s anymore; stop using shared cookies as SSO! All an attacker has to do is acquire a cookie, and they become a legitimate person in the organization, free to wreak havoc.
There’s a particular vulnerability that is an attacker’s dream, where the *.site.tld domain is deleted, but its C record in the DNS isn’t. An attacker could register that subdomain and gather cookies, and eventually, lots of organization data:
When it comes to OAuth, you’re looking for implementation vulnerabilities, in either the client application, or the OAuth service.
In the OAuth flow, only the IdP (identity provider) holds the user credentials, which are contained in the ID token. As an attacker, you want to somehow steal the ID token, which you can then use the request the access token, which is the key to the resources you want to get your paws on.
Because of its delegated nature, OAuth relies on open redirects. A poorly-built or -configured OAuth service that fails to use a list of allowed redirect URIs could be exploited, but that’s the sort of thing that Auth0 doesn’t allow.
As far as CSRF (cross-site request forgery) attacks are concerned, they can be mitigated with OAuth 2.0’s state parameter. For each authentication request, set it to a hard-to-guess value, and see if the response is the same as the one you sent with the request.
And of course, there’s always checking for bad implementations of the standard:
Here’s another meme I’m going to share on the Okta Slack:
And finally, there’s SAML. As the mobile specialist for Auth0, I never touch the stuff:
But if you’re doing pentesting on a SAML-based setup, you’ll want to use SAML Raider, which add SAML-specific functions to Burp Suite:
Last presentations of the day
I caught a bit of Dan Fernandez’ presentation, The Boring Parts of AI: Risks and Governance of Large Language Models — you can find the slides here…
…and a sliver of Cochise’s How to Wage War and Bypass Congress: a Primer on Gray Zone Warfare preso, because it’s always fun to see him go off on a rant.
Thank you, BSides St. Pete!
To Wilson Bautista and the BSides St. Pete team, my thanks for a great event for the cybersecurity community to share knowledge and gather together!
In all my years, I’ve failed it only once. But I’m certain that actually experiencing that failure ensured that the lesson would “stick.”
I got the usual “Your manager will be notified and you’ll have to undergo mandatory security re-education” message afterward. Surprisingly, my manager never brought it up, and I was never scheduled for the “Don’t do it again, dumbass” remedial course, but believe me: I learned my lesson that day.
