Thoughts on the OST2 "All-you-can-learn buffet" in-person training format

By Xeno Kovah, Founder of OST2

Background

I want to teach in-person computer security classes in a non-standard way. Because I think I've found a better way. 

I have been teaching in-person live-lecture style classes on things like x86 assembly language since 2009. At the time I worked at MITRE, which had an incentive program where employees could get a $2000 bonus for teaching a 2 day class to their colleagues. As someone only a couple years out of grad school, that was a monumental amount of money to me, so I then went on to develop other classes on things like x86 OS internals, Windows rootkits, and binary executable formats. Internal recordings and slides from these classes, and others from my MITRE colleagues, went on to be the initial content for OpenSecurityTraining.info (OST1) in 2011 after we got them through the public-release process. By the end of my MITRE tenure in 2014, and into 2015 as I co-founded LegbaCore with Corey Kallenberg, I was doing numerous live-lecture trainings at security conferences and privately for companies.


I tell you this to say that I have done a lot of training, and have a fairly good sense of the pros, and cons, of the traditional live-lecture class format. And while other OST2 instructors may choose to teach in-person classes the standard way, I personally want to teach in a non-standard way.


Business Models


Obviously the first thing that's non-standard is that with the setup of OpenSecurityTraining2 as a formal non-profit in 2021, the primary emphasis is on providing the world's best cybersecurity education for free. Not low-cost, not freemium, but free. Both free as in beer, i.e. open access where anyone can take the classes whenever they want, but also free as in freedom, i.e. all material existing in editable form under permissive open licenses like Creative Commons.


But the most successful open source projects need more than altruism to support them. They need a business model. If I look at the growth of Linux, I attribute it in large part to corporate contributors who were acting in their own best interests. According to the 2020 Linux Kernel History Report, the top 20 companies accounted for 68% of all commits. This is why for OST2 our primary model for growth is to work with sponsoring companies and governments to show them how support for OST2 is in their own best interests, both for acquisition of new talent and skilling up their existing people. But there's another famous open source business model, popularized by RedHat: making the product free, but charging for support. And that's where I think OST2 intersects with the traditional in-person classroom delivery. Except that I want to teach computer security classes in a non-standard way.


Very quickly as college professors tried to translate the Khan Academy model to their own Massive Open Online Classrooms (MOOCs) around 2012, they found that the more difficult the class, the more difficult it was to keep up with student questions. Also, any sort of exercises that didn't lend themselves to programatic grading, simply couldn't scale. They tried different strategies such as recruiting armies of TAs from their colleges, or trying to make students grade each others' questions. But simply put, as far as I can tell, no one has yet solved the problem of support-at-scale.[1] So that brings us back to how we can support people who really want or need support in pursuit of their learning goals.


To me, in-person classes, already such a widely used training mechanism in the security community, makes sense as a natural fit for the RedHat "pay for support" model. I explain below why if you swap out the restricted-content, live-lecture material of a traditional training, for the free-content, recorded-lecture, live-support approach of this new format, you end up with a much more beneficial training for the student.


[1] I'm vaguely optimistic that someday things like IBM Watson will come down in price and complexity to the point where it can start to be used for things like answering the most common questions encountered in computer classes. One can imagine the training of such virtual tutors being based on consuming forum posts such as those on StackOverflow, which have correct answers marked by the question-asker.


What is AN OST2 All-you-can-learn Buffet CLASS?


It is the delivery of multiple classes from a given instructor at the same time in the same place. Instead of the instructor delivering an in-person lecture, students are provided videos with subtitles for lectures and lab exercises which they can watch at up to 2x speed. Students watch the content for a given class at whatever speed they're comfortable with, and the instructor is on standby throughout the class to answer questions as they occur. Students can raise their hands to talk in person, or ask questions on public or private chat rooms set up for that specific class. Because the instructor is not spending their time providing a synchronous live lecture, they have more time for answering student questions.


What makes it an all-you-can-learn buffet however, is the fact that all these classes from the same instructor are likely related to each other. Most often it will be a sequence of ever-more-advanced classes, which depend on the prerequisite knowledge from the earlier classes. Students who know earlier prereqs can skip ahead, and keep progressing through as much of the class material as they can achieve in the given time. Or students who need to start from the beginning can do so, and complete as much of the material as possible. But because these are OST2 classes, they can just keep advancing through the online classes even after the in-person venue is finished. And because they're paying for the support, not the content, the instructor will continue to provide some level of support for answering questions and helping with labs and exercises even after the in-person venue is finished.


Cons of Traditional PAID TRAININGS


Let's talk about some of the disadvantages of a traditional paid training format, and how we can solve them with an OST2 "All-you-can-learn Buffet" class (hereafter "OST2-B class").

Problem 👎: People forget the material shortly after class. This is expected. But typical classes leave students with little more than a copy of the slides to try and remember the content by.


Solution 👍: Give the students videos of the lectures! It sounds so simple, but of course with a traditional class where the student is paying for the content rather than the support, the instructor would be very very worried about giving their class material to students, because it would inevitably leak, and then their value proposition would be destroyed. At OST2 the videos are already online and freely available, so we have nothing to lose, and everything to gain by students having access to the full lecture & lab videos after class. This help ensure that if they want to refresh themselves on the material 6 months, a year, or 5 years later, they always can.

Problem 👎: Instructors at conferences get paid per-person. This means they're incentivized to fill the room with as many people as possible. Because the pool of potential students is largest at the level with the least prerequisites, they are incentivized to teach primarily intro-level classes, because that's the easiest way to fill a classroom. Thus the security community suffers for lack of more advanced classes, because instructors have a much harder time filling those classes. This situation is exemplified by the below pyramid. There are a notional 21 students available at the introductory level, but only 9 at the intermediate level. And while some conferences will teach a class for 9 students, some won't. And none will teach a class for only 3 students.

Solution 👍: The above is what the OST2-B class structure is trying to help solve. Thanks to the asynchronous delivery offered by recorded content, in an OST2-B class, students from multiple classes can coexist in the same physical (or these days, virtual) classroom. 

Students can start out in whichever class they signed up for, and then progress through it at their own pace, asking the instructor questions immediately when they need help. But most importantly, if they're one of the fast students, they can move from one class, for instance x86-64 assembly language, into the next class, for instance x86-64 OS internals. Or from x86-64 OS internals, to x86-64 reset vector firmware. Because the classes are already arranged into learning paths, it's understood which classes are prerequisites for which other classes, and thus where a student can go from any given point. And again, because the students in in-person classes are paying for support, they can get that support both in class, but also after class. (Currently I'm starting by setting the support level for my own classes at answering up to 20 questions within 3 months of taking the class. By setting a time limit for the support, I hope to encourage students through this use-it-or-lose-it incentive to keep progressing along the learning path quickly after class using the free online material. We'll see how this initial support level agreement pans out out, as it may need tweaking. And other instructors may set other support levels based on my or their own experiences.)

Problem 👎: There's another problem that comes from the incentivization to teach primarily intro level classes. And that is that instructors who want to teach more advanced classes end up being tempted to not exclude potential students by claiming they will cover prerequisite knowledge in class. I often see this with classes on exploits or reverse engineering where an instructor acts like they can teach the basics of assembly language in the first few hours or day. This being in contrast to students of my assembly language course generally taking 20 hours or more, not including the very intensive reinforcement lab exercise which can take over a day itself!


Solution 👍: OST2 itself, actually helps deal with this problem in some cases. Rather than trying to quickly skim through prerequisites, instructors can link to our free classes and tell students they need to know that material before they arrive in class. But OST2-B classes solve this problem by not needing to make material a strictly "do this before class" prerequisite. They can make prerequisite material into something which students can optionally register for if they want to go through it all, or which they can pick and choose bits of as needed.

Problem 👎: Even for instructors who do the right thing, and list content as true prerequisites rather than trying to skim through them in class, students will still inevitably show up to class without the necessary background.


Solution 👍:Like the previous problem, this is again where the unique OST2-B class structure is resilient against missing background info. Let's say a student registered for my more advanced x86-64 reset vector firmware class because they thought they knew all the prerequisite material. But then once they get into it, they find out that they didn't really know 32-bit segmentation as well as was needed to progress. In a traditional class, the student would suffer from confusion which might resolve itself, or which might persist throughout the class, leading to a suboptimal learning experience. In an OST2-B class, they can just quickly dip back into the videos on the prerequisite topic area, and then resume learning the material which depends on it. 

This would never be possible with a synchronous live-lecture class. There's always going to be a level at which an instructor doesn't cover the prerequisites themselves. But because OST2 is about building deeper training, we're more focused on the case of more advanced OST2 classes being built on top of less advanced ones. Which means that for advanced classes, one can always go back and quickly apprise oneself of only the necessary background material by watching only the relevant modules of some prerequisite class. (Which the instructor can directly point them to when they're in an OST2-B class.)

Problem 👎: Different students have different expectations about the relative proportion of lecture vs. labs in classes. While it's known that hands-on labs are superior to lectures for knowledge retention, if a student comes to class and it's almost all labs, they can leave feeling that the instructor was just trying to fill the time, without having to spend much time actually teaching. On the other extreme, if it's all lectures, the students can feel like they got all theory and didn't learn anything practical.


Solution 👍: Having the class pre-recorded lets the students adapt their in-class time to their own learning tastes. If they're just there to get exposure to the topic, but unlikely to be doing the hands on work in the domain themselves (e.g. if they're a team lead or manager), then they may choose to skip many exercises, and just go through as many lectures as they have time for, preferring to cover everything they might want to ask the instructor questions about. If on the other hand they know they're going to be using the material immediately in their work, then they may want to really drill down and be able to complete all the exercises while they have the instructor available to sort out any issues with lab setup or instructions.

Problem 👎: People complete labs at very different speeds. In a live-lecture class, this leaves more experienced students sitting and waiting for the class to continue, and leaves the inexperienced students being forced to move ahead before they've finished the labs. This deprives the latter group of the feeling of accomplishment which is necessary to ultimately feel that they enjoyed the material and want to go deeper. Indeed it can give them the wrong impression that they're not good at the material, when it simply might have been the case that they happened to be in a class with people who already had some experience with the topic. Instructors in live-lecture classes always have to try to strike a balance between the needs of the experienced students, and the inexperienced students, and it's almost always going to leave some students with a suboptimal experience.


Solution 👍: The solution to this is the same as the solution to the previous problem. Having the class pre-recorded lets students who choose to do all the labs take as much time as they need to ultimately complete all the labs they want to complete. In a live-lecture class the instructor may have made some optional labs, more to keep the fast students busy than anything. But in a pre-recorded class, the instructor can make plenty of optional lab material, for those who want the class to be more hands-on. This once again lets the student knowingly tailor their classroom experience to their preferences, instead of being locked in to the one-size-fits-all class.

Problem 👎: People are sleepy after lunch and have a hard time concentrating! This is most often a problem at major international conferences, where the attendees are often coming from far away and are jet-lagged. Having been both the student and instructor in these circumstances, I will make the bold claim that if the student is sleepy, they're not going to absorb the material as well! ;)


Solution 👍: Another unique advantage of the asynchronous delivery of an OST2-B class is that students are free to get up and take a break whenever they want. They can stand up and move around to get the blood pumping again. They can get a snack or some coffee, or just go chat with the instructor if they're not answering someone's questions. All in all it leads to a less rigid class which is more adaptable to students' needs, which leads to better learning outcomes.

Problem 👎: Students might be embarrassed to ask questions in general, or in front of their colleagues who are also taking the class. Or they might not want to interrupt the instructor right when the question popped into their head (at which point they stop listening, and start repeating the question in their head trying to remember it as they wait for a pause to ask it.) Or they might adhere to the common practice of not asking a question because they think they'll be able to figure it out as the instructor keeps talking and gives more context.


Solution 👍: Once again the OST2-B class structure offers a significant advantage by letting students ask questions immediately when they have them, without fear of anyone judging their question. In a typical in-person classroom setting, this can be the traditional raising one's hand. But it can also be asking through an in-class chat, either on a private or public channel, based on their comfort level.

Problem 👎: Any instructors who don't teach full time will give their classes to a fairly small audience. Even if they're an excellent instructor, this means that the number of students who look at the material and can spot errors is fairly low. Some subtle errors can take many hundreds of students going through the material before they get pointed out.


Solution 👍: Because the classes that students take in an OST-B delivery are always available online, they get a much much larger audience looking at them than traditional classes. While most of the errors end up getting corrected during the private beta testing before wide release, as mentioned, some errors simply need many eyes before they can be seen. This means that in general, students of OST2-B classes are benefitting from much more thorough error checking than traditional classes. And the errors can always simply be corrected with easy website updates and video edits, so that students can be confident they're getting an extremely low error rate. This is not confidence that can be achieved in traditional classes, unless they've been taught for many more years by full time instructors.

Problem 👎: Sometimes it can be difficult to understand what an instructor said. In particular for English as a Second Language (ESL) students listening to ESL instructors. And while of course a student always has the option to stop the instructor and ask them to repeat themselves, if they're having persistent difficultly understanding, they're going to be much less inclined to do that over and over, for fear of inconveniencing the class.


Solution 👍: At OST2 we make sure all our videos are captioned, for increased accessibility and easier understanding of instructors. Because we have ESL instructors and ESL students, we know that it's always appreciated to be able to double check what specifically an instructor said. And we also put our subtitles into git repositories allowing the community to submit corrections whenever something seems to be in error. So our captioning is only getting more accurate over time.

Problem 👎: Paid trainers often travel to conferences all around the world, getting jet-lagged in the process, which can affect their teaching quality.


Solution 👍: A pre-recorded instructor is never jet-lagged and always full of energy! Also, live-lecturing for a full day can be surprisingly exhausting, and thus by offloading that task to the recordings, it fully reserves the instructor's mental energy for answering student questions.

Cons of PURE ONLINE TRAININGS


Pure online class delivery is not without its problems too. The most notable is that many people start classes, but few finish them.


Problem 👎: If the student is particularly busy with work or school, it can be hard to find the time to really get in the zone for deep learning. And going through the material only a couple hours at a time, at night, after one is already tired, can lead to needing to repeat some material in order to fully absorb it.


Solution 👍: Attending an in-person training serves as a forcing function to make oneself get through the material, free of distractions, and with the full energy of a workday.

The challenge and opportunity of teaching a better way


In the above I outlined many of the advantages that I see to the OST2-B class delivery. But I would be remiss if I didn't mention the one major challenge that I see to OST2-B class delivery: people's expectations. It's not what people expect out of a typical class. Some people just want to come to a class and have an instructor speak at them, and to be able to speak to the instructor at will. Where speaking to the instructor is concerned, hopefully I've shown that with an OST2-B class the opportunities to do so are far more than they are in a traditional class. It's literally the instructor's whole job during the class!


But there might be this nagging feeling that they're not getting their money's worth if they're not getting a traditional class. Or, more insidiously, they might see an OST2-B class and think to themselves: "Oh, it's freely available online? I'll just take that class later on my own time then...". And while I don't mind conference training pages serving as an advertisement for OST2, the thing is, I think that the sets of people who take paid trainings and those who have time to take free classes online is largely disjoint. People who take paid training typically do so because they have more money than time. And they want to save time by getting the much quicker guided tour of the topic area, rather than being forced to take the meandering self-guided tour of looking through whatever material they can find on the internet. 


So this will certainly be a challenge - explaining, in a more concise way than this post, to everyone who happens upon an OST2-B class, that they're actually looking at a better way to learn. But here again the OST2-B format has one last trick up its sleeve... Even if prospective students end up deciding to take the classes on their own time instead of getting live support, we still have the potential to fill all the seats for in-person classes from a subset of the total population, like shown below. And the deeper our learning paths become, the more potential audiences we'll have, and thus the easier time we'll have filling classes.

Conclusion


I want to teach computer security classes in a non-standard way, because I believe it's a better way. So I'm going to be evangelizing and trying this style of teaching out for the next couple years, hoping to blaze a trail that other instructors can follow. Because the more instructors we can show that they can still make a living teaching open security training, the more material will become available for everyone, and the faster we'll get to the future.


Thanks for reading,


Xeno

Future OST2-B Classes


The following are upcoming classes which will be taught in the OST2-B format:

PAST OST2-B Classes