Dacia Toll | Coursemojo - Transcript

Jeremy Singer: [00:00:00] Dacia, thank you so much for making time and joining the Education Equation podcast. Dacia Toll is the co-founder and co-CEO of Course Mojo, which offers a curriculum aligned, AI-powered teaching assistant. Dacia also co-founded Achievement First, which is one of the leading and most innovative charter school networks with roughly 40 schools across the Northeast.

Dacia, it's a bit crazy to think we met over a quarter century ago, and it's been thrilling for me to watch you as you built one of the most impactful charter networks and achievement first, and now we're going in a very different direction, but also very impactful with Course Mojo. There is a ton we can discuss, but let's agree, today we're gonna focus on Course Mojo, I'll keep an option to dig into achievement first another time.

Sounds Uh, so with that, let's jump in. Help our listeners understand why you founded Course Mojo and what specifically it is designed to address. 

Dacia Toll: We are going after two big challenges um, with our school [00:01:00] partners. The first is reading Achievement, um, which we all saw the very sobering NAEP, uh, or national reading data.

It's at a 30 year low for where eighth graders are as they head off to high school. And so while we know students took a big step backwards during the pandemic, frankly the decline was happening long before that. And it is now, especially the number of students that are scoring below basic, it's a national emergency. In terms no question of the implications for young people in the rest of their lives.

Um, so that's the first big one. The second one, and it's related, is teacher retention. Um, we have got to figure out a way to make this a profession that more people want to join and stay in, uh, and get ever better at. And so those are the two biggies. 

Jeremy Singer: That's awesome and two incredibly important, uh, challenges facing all of us and all of education today.

Um, why don't you give a quick sense of like how course Mojo works and how it addresses those two [00:02:00] pieces. 

Dacia Toll: One of the biggest differences between what we're doing and what so many others are doing is we are not a supplemental. Um, the vast majority of EdTech productsUm, are kind of something extra that teachers have to squeeze in somehow for 20 minutes a day, right? Or, or twice a week. And we are trying to make the core instruction that teachers are already trying to do for the day, the ELA block, the English language Arts block. Um, we're trying to make that more effective,

more engaging and easier. And so we are aligning to the high quality instructional materials that a lot of districts and schools have adopted. Right. Um, and what we're doing is embedding the AI supports for students and teachers in those materials. Uh, that's 

Jeremy Singer: great. And I wanna jump into the specifics, but I just wanna also underscore, uh, you know, when you're designing a product, you always wanna understand your users and what's, uh, frustrating?

In my position, I see so many startups that don't talk to teachers or don't really [00:03:00] understand what teachers need, and they develop these supplemental products. And I think both of us know You talk to the vast majority of teachers, the last thing they want is something else that they have to fit into their schedule, their school day.

They already are overwhelmed with what they have to do. And so I love the fact that you've really pinpointed and anchored in on, uh, trying to provide something that helps them do what they already do more efficiently. And I'm sure that helps with the teacher reception. Tell us more about how it works.

Dacia Toll: Well, what that looks like then, there's both student facing supports and teacher facing supports, and it all works together. So what that looks like from a student perspective, and this gets back to another big difference, is the vast majority of EdTech tools are also silent solo experiences. And so kids put on their headphones and they go down their own personalized pathway.

That's not this. Um, we actually want students, um, working together in partners with Mojo, uh, Mojo's, the AI powered teaching assistant. Yep. Um, and so it's like you have a third learning buddy in your group. Uh, Uh, it's often [00:04:00] partners, sometimes more than that, and students, because we're anchored in the curriculum. It's the same rich, wonderful, complex text that they were already reading that day, the same thinking questions that they were already grappling with. Um, but now as a student said, it's like the handout is talking to me. Um, and I love that. So they talk to each other and they talk to Mojo and they go from, you know, mojo categorizes student learning, but sometimes they start it confused, but trying, so it's not, not an effort issue.

Right. But they're, they're not sort of fully getting the sort of deeper meaning, the metaphorical meaning of a poem or the um. Author's craft of a novel or the big ideas of a nonfiction, and then mojo nudges them and they support each other to go to partial understanding and then to full understanding.

And so while that's happening, while students are working together. Teachers have a live dashboard that shows them every student's level [00:05:00] of understanding for every single question. Um, and many have described that as a game changer. If you've got a room of 25 to 30 middle school students and you can quickly see who needs your help and what they need your help with, it just helps you to be so much more strategic with your conferencing.

Then after students work together for 10 to 15 minutes, um, the teacher will often pause the class. And then go to their, to mojo. And with a click of a button, it will tell you the two most common misconceptions in your class right now. Um, and teachers review those and Mojo suggests potential discussion questions and they make the choice, but now they're doing it in this wonderfully data informed way.

Jeremy Singer: Yeah, I, it's great. And, uh, as you know,  um, uh, 20 plus years ago when we, uh, David Coleman and I, uh, started Grow Network, a lot of the same concepts was trying to take data, this was from the summative exams, and then feed it back to the classroom teacher so it could actually inform instruction. Uh, it was very rudimentary compared to what you're doing.

I think one of [00:06:00] the challenges we had, um, and I'd love to hear how your experience has been, is we continually overestimated,  uh, uh. The complexity of the data. Like we, we provided a lot of data and it was just too much. For a lot of teachers, again, who have so many, it's not that they're, uh, not facile with data, it's just they have so much stuff they're trying to do.

 Uh, and so we had, I think, an overly ambitious idea of how they could apply the data. And so over our time, we just kept simplifying, simplifying and narrowing. But like, what have you found and how do you deal with that challenge? 

Dacia Toll: So I think the big game changer for when you and many others were working on these same sort of problems and opportunities and now is ai.

AI can do a number of things. First it frees us from a world of multiple choice. So the kinds of questions we're able to ask, the kinds of student responses, um, we're able to go after is, um, we can now ask students what we want to ask to Almost all classroom questions are a wonderfully open-ended question.

Right. Right. And the AI can evaluate those [00:07:00] responses. The second thing is it can analyze large amounts of unstructured data almost instantly. 

Jeremy Singer: Yeah. 

Dacia Toll: Um, and so you always had I think after the fact. Yes. Uh, part of it. So what we are trying to do is, in this moment, what is the teacher trying to do?

Um, when I was at Achievement first, we used to call it rapid feedback. Different places call it different things, but when students are working, you as a teacher are circulating around the classroom and trying to both support individual students and then figure out what's the trend. Um, and student misconception and it's, it's wonderful and it's hard to do, um, particularly in a class where you've got, you know, sometimes unfortunately management concerns have to be the biggest concern.

Um, so the good news is mojo's so engaging that students are working effectively with their partners, but then it really tells you, so it's in that moment, what's the information I need to take? The action I want to take? Yeah, so it's very simple colored dots. That a teacher drew for us. Uh, yeah. In one of our earliest [00:08:00] pilots.

Uh, back to your point about the importance of teacher feedback and then it's the top two misconceptions. We could give you seven, we give you two. 

Jeremy Singer: Right? 

Dacia Toll: Um, a suggested discussion question for each, and they're able to make that real time choice about what would be best. 

Jeremy Singer: I love it. Um, and now I'm thinking back to like, business school of, we studied just in time manufacturing and this is sort of just in time instruction, which is, is brilliant.

And that was a model we couldn't do when, you know, after the fact. So I think the timeliness makes a ton of sense in driving success. Um, that is awesome. So , Um. What has been the teacher response? I mean, has, has everybody embraced it?

Have you got challenges? How has that worked? 

Dacia Toll: The overall response has been very positive. I mean, we have gotten critical, constructive feedback and that's made us so much better. But, um, we survey teachers regularly. We talk to 'em informally all the time. Um, but in our surveys last year, the average teacher rating across a number of dimensions was a 4.2 out of five.

That's [00:09:00] awesome. Which for our first full year was very strong, not just in terms of um, how much they think it's supporting student learning, but to that second goal we have, how much it's making their jobs easier and more enjoyable, how much it's helping them implement the core curriculum. Uh, Uh, so that's been very positive.

And as I've said, it's every step of the way. It's teachers and students who've said to us, Hey, could, but this is good, but could you do this? Or. This thing's a little frustrating or takes me a little too long, or I think there's a better way, and those have been the breadcrumbs that we just continue to follow to make this even better.

Jeremy Singer: Yeah. No, it's awesome. And I, I feel that your, uh, you know, instructional experience and being in the classroom, developing schools and being a school leader, you know, really it allows you to do things very differently. And I am always nervous. I mean, there's a lot of successful people coming in from outside, but that experience is critical. I also love how you're bringing up the 92% and you know, one of the things that I appreciate around your approach is, um, and we're, I think we have a good [00:10:00] kinship is measuring and using data to evaluate impact. So I know there was recently, uh, some good data that came to you about, uh, the impact.

Can you give us a quick, uh, summary, uh, of what it was and the results. 

Dacia Toll: Yeah, impact is always most important. Um, it is interesting. I find myself being fairly critical of the education technology industry. I've been a skeptic for a long time and I think it all sounds good on the front end uh, but routinely people over promise and under deliver.

Yeah. And you were 

Jeremy Singer: a recipient, I mean, running achievement first, you were obviously trying different things and so you saw firsthand that a lot of the promise didn't actually, uh, turn into reality when you adopted. 

Dacia Toll: Yeah. And so we were holding our breath, uh, for ultimately state test scores because you see progress along the way, but sometimes it doesn't translate and it has translated substantially. Um, so, uh, two of our biggest partners have gotten their results back and Aldeen, ISD in Texas, um, they had the largest reading gains of any, um, large district, of which they're like [00:11:00] 115 in the state of Texas. More than 10,000 kids in the state.

So they really blew it out. Now part of that is credit to them. They had, um, done some really thoughtful work on coaching. Um, they've adopted HQIM, but then there was a distinct mojo layer. Uh, and so they piloted Mojo in some middle schools and not others. And the schools that piloted Mojo on top of the district wide gains had another six percentage points.

Wow. Which may not sound like a lot, but for that, that does sound like for middle school reading achievement, um, that is, that makes you, uh, the most improved in the state. Uh, it is hard to move because reading, as you know, is so complex. Yeah. Um, and then Sumner County and Tennessee, um, Um, similarly, they had a dozen middle schools.

They had six use Mojo six, not the schools, uh, and students and teachers who did, had an eight percentage point gain. Wow. Um, Um, congratulations. 

Jeremy Singer: Yeah, that's awesome. Thank you. 

Dacia Toll: and we're doing a variety of research studies, um, and they have told us that that meets the ESSA tier two standard for evidence of impact, which in our [00:12:00] first year, um, we're certainly excited about.

Jeremy Singer: That is awesome. You know, we talk a lot about, uh, I think maybe when you mentioned your frustration with innovation and really, um, interventions or programs that change student outcomes. And there's again, so much hype, but so little, uh. Uh, results, which is part of why I started this podcast. Um, but, you know, part of it's a challenge, right?

It's hard to do a randomized control trial in a school. There's just so many factors that you can't control. So to have a test and a control environment's hard. But I love it and, well, it's not perfect. I love that you have in both these districts. A set of schools that, you know, were course mojo was being used and then a set of schools that weren't.

And you can see the differential gain. So, um, uh, you should be super proud of that. And, uh, I'm sure excited. So, um, take us through sort of what's next in the evolution. You've got something out, it's already having some impact. Uh, how do you see it expanding? How do you see it growing?

Dacia Toll: Well, we're certainly adding more grades, more curricula that we support. So we were aligned to three, uh, curricula last year: wit and wisdom, EL, [00:13:00] and my Perspectives, and we've now expanded that to a list of seven. Um, uh, and then we've gone from, uh, one grade to many grades. Uh, focusing really though on that middle school, uh, challenge where.

Um, after the work, the very important work on phonics, I fluency sort of the bottom half of Scarborough's Rope and the science of reading. Yeah. Kids hit what's been called a comprehension wall. Um, and making that next jump up to really being able to think critically about the text and understand increasingly complex ideas.

Um, it's a thinking job as much as a reading job. Um, and so that's the part that we are focused on. So we've done that. Um, and then in terms of the product itself, we maximized first for the everyday classroom experience of students and teachers and making sure that we were, um, adding value in that moment, helping teachers not just save time, but be more effective.

Have more teachers able to execute the moves of effective teachers. Um, and then now we're shifting to expanding [00:14:00] back to the power of AI and the ability to analyze student work almost instantly. It doesn't just have to be today's work. It can be over time, which will enable us to see trends in student learning over time.

Um, and then we are adding coach tools as well. So beyond just the individual teacher, how can school leaders, um, and coaches have visibility into that student learning and teacher practice? 

Jeremy Singer: It's awesome. Um, really inspiring. You know, I love also so many parts of this, but that you're focusing on middle school.

Uh, and you laid out this sort of literacy development, uh, so well, but the other thing that is a challenge is a lot of people have like given up on any progress in middle school. You know, there's, there's elementary and then there's high school, and people think this is just a way station.

So I love that, That is one of the hardest places to tackle from my perspective at College Board. Uh, what's interesting is we. Uh, as you know, like in AP exams, one of the best parts of the exams are the document based questions, the DBQs, which really force [00:15:00] a student to analyze multiple sources.

And these can be texts, they can be pictures. They can be letters, and then synthesize the meaning. And I've always thought if we could bring DBQ into middle school, that would be another angle exactly. Aligned with that. Um, I wanna go back to something you said, 'cause a lot of the listeners as all of us are very interested in AI today.

And you mentioned two great benefits of ai, which I don't think everybody thinks about. But one is that you can go well beyond multiple choice. You can start to do, um, uh, longer writing, um, uh, work, which is great. And then I also love the idea of its ability to analyze vast amounts of data.

 We're just at the tip of the iceberg on that. But on the, the first part of analyzing writing, um, and giving writing feedback, you know, tell me how you do it. Do you train a model on these prompts? Have you got to sophistication where it doesn't need training?

Like, just help our listeners understand literally how that works. 

Dacia Toll: Yeah, one of the things that we're doing differently that frankly takes a lot of time, is for every single question we [00:16:00] are telling the AI what the levels of understanding are, what constitutes criteria for success? What constitutes, uh, a sort of level 1 0 1 understanding the variety of answers that would do that.

A level 2 0 1, 

Jeremy Singer: You have some sort of rubric but is it specific to each text? 

Dacia Toll: It's specific to each text and it's specific to each question. 

Jeremy Singer: Oh, wow. So it's a lot of work. Yeah. So 

Dacia Toll: I think part of it is a lot of tools out there are these generic AI bots that maybe have been tuned and trained, but I think when you go beneath most of them it's just prompt engineering.

Um, and the AI is powerful and it will give you Uh, an acceptable answer some amount of time, some amount of time. It will give you an incorrect or unacceptable answer. Right. Right. And when you're working with struggling readers, our tolerance for answers that are not accurate is, uh, pretty low.

Um, 

Jeremy Singer: Yeah, and I don't think a lot of people understand that we're, you know, in many cases, , uh, if there's some random wrong answer, it's okay. When you're doing [00:17:00] You know, a search online or you're doing something, but in, in education, it can be quite impactful 

Dacia Toll: And it's not, I think the large language models themselves are wired to be our, uh, assistant and to tell us the answers with a lot of confidence, right?

Whether they're, um, and they're, they are working on increasing like a few 

Jeremy Singer: people we know, but 

Dacia Toll: Yes, that's not what we need in an educational setting. We need a sophisticated tutor. Um, and that's not giving you the answer that's asking you the just right question um, that doesn't drain the rigor and the thinking out of it.

And so, um. these texts are nuanced and sort of what the level of understanding is. So now we are increasingly using AI and human authors to author the criteria I'm describing to you. So the texts and, many of the questions, the initial questions themselves come from this core curriculum. Um, but what we are using the AI to do is to develop all of the criteria for each question.[00:18:00] 

Then the scaffolded questions, and then the sort of transferrable takeaway. Because the other thing is we get to read this wonderful text today and we get to spend time grappling with it and getting to deeper levels of understanding in a way that really can be fun. Um, and then we need to understand what we did today that's transferrable, um, as effective readers and writers.

So, 

Dacia Toll: That's the way in which we're using the AI is, it's to categorize that open-ended response. And as you said, we can do longer form writing, but the other big change is it's almost like we can have a conversation with the AI. Like Yeah, in the initial part of the activity, there's a lot of fast passing.

Between the students and each other and the AI and it feels like we're just having an informal conversation in which we're grappling with what is this text trying to tell us and why, how is the author going about it? And then we have a class discussion with the full group, and then we come back and often it ends with a writing piece.

Jeremy Singer: Right and you've talked about, we've all talked about this [00:19:00] sort of Socratic method that. Uh, was once very much in style in real life. The sort of fell out of favor for, for a myriad of reasons, but seems like it's having another, uh, run now with, with ai. 'cause AI can be this sort of partner in that Socratic discussion.

You're using ai, it's literally helping the student along in their thinking. Right. Yeah, 

Dacia Toll: I could talk about, get in the weeds and talk about all the specifics. All day long, Jeremy, so help me here. But when I think about what's driving the results. The biggest thing is kids are doing more thinking, more reading and more writing.

Jeremy Singer: Right. 

Dacia Toll: Um, and time on task. 'cause we've started with high quality instructional materials. Like if we weren't starting there, this would be a different conversation, but because we're starting with text worth reading and questions worth asking,  um, and because mojo' favorite move is actually to direct students back to a particular part of text.

Jeremy Singer: Yeah, 

Dacia Toll: um, to a particular paragraph, a particular sentence. And get them to keep doing that thinking. Yeah. And because kids like mojo and he, when they [00:20:00] send them a message, he'll give 'em a little emoji, thumbs up, a dance guy, and you know, there's a ever so slight gamification mostly around praise for authentic student work.

Jeremy Singer: Yeah. Effort. 

Dacia Toll: Uh, but it's very subtle. I don't believe in dialing that up too much, but the kids like it, and so they like the overall experience. One of the things we're proud of as teachers say, they're more likely to do partner and small group work in class. Yeah. Because they now have eyes everywhere with mojo.

Yeah. Yeah. So it's more social, actually more discourse and more motivating. 

Jeremy Singer: Yeah. Yeah. 

Dacia Toll: And so then the kids are putting forth more effort and because they're receiving up to three rounds of feedback on every question, like exponentially more feedback than would be possible otherwise, that effort is increasingly effective.

Jeremy Singer: what's, uh, so exciting is you, you have a very, uh, intentional instructional, pedagogical approach, but you've also thought about the engagement emotional side and you know, I've lived at places [00:21:00] where we've had great pedagogy, but it couldn't engage users, or we really can engage users, but the pedagogy is weak and so you seem to be balancing that line.

I agree with you. I think gamification is another area that's been way overstated. Uh, but at times it can work. I always worry it's like a fad. When my kids were young, they used to do the streaks on some of the apps where they'd post something every day and have friends send a message.

I don't even know what it was. Um, probably a bad parent.  Uh, but I watch Duolingo and uh, my kids are older, but I see younger kids who want to keep streaks alive or do something and they've really gamified. I, I don't know if that will stick. 'cause I think. Those trends, if you look, uh, you know, Snapchat, there were streaks in Snapchat and then it disappeared if you look on the presence on Google, some of these things, so I, I do think

uh, over align on that is questionable. I wanna do one more question then, then I wanna move on to something else. But there's a lot of, uh, scaffolding you're providing to the AI to help the student through the work, the speed at which AI is improving, I mean, it's mind boggling.

Are you thinking there's a world where you're not gonna have to, [00:22:00] like, where the AI will get to a point in your confidence where the amount of work to prep a particular question or particular prompt the amount of pre-work you'll have to do, will, will be reduced. 

Dacia Toll: Um. I hope so.

It is getting a lot faster already because we're using AI to help us set the criteria Right, right. For other AI to use, to evaluate. So yeah, I'm certainly optimistic that that will help. Um, and I think there are a number of different solutions that kind of layer the AI onto itself to do some self checks.

Um, we also now have, um. Before any activity sees live students, we have a whole bunch of AI students basically, uh, push on it and test on it. Yeah. And optimize prior to that. And then of course, once we go live with students, they give us good feedback too. Um, so I think there are a lot of ways in which it will become more efficient.

I am skeptical that we will reach, um. A truly strong [00:23:00] pedagogical ai. Yeah. On its own. 

Jeremy Singer: Yeah, 

Dacia Toll: because I don't think it's being built for that use case. Yeah. Um, I think it's being repurposed from, uh, uh, this as we described, sort of more assistant expert use case. So, um, the ability to make a key point about the difference between an inference question and a right there.

 I think there's just a number of ways. Yeah, I would love to be pleasantly surprised, but our assumption, um, is that to your point, you have to really dig in and maximize for strong teaching. Strong teaching support. Right, 

Jeremy Singer: right, For 

Dacia Toll: the teachers and for uh, students. 

Jeremy Singer: I think I'm a little more bullish.

I agree with the point. It's not designed for some of these things, but. I've watched AI, using it for things that definitely weren't designed for being increasingly effective. So, look, you said it'd be a pleasant surprise. Let's hope you get that surprise. Um,

Dacia Toll: one thing I will say Jeremy,

to your point, it just does get so much better. Yeah. So much faster. 

Jeremy Singer: Yeah. 

Dacia Toll: Um, we haven't talked about [00:24:00] this yet, but one of the things that's possible is. Um, real time translations, not of the entire experience, but in a targeted way to help students understand a particular question, particular piece of feedback.

 Um, it can, of course, translate the whole experience. That's all ai. People are blown away when there's the list of 65 different home languages. Um, that's the AI. And speech to text. And text to speech. Yeah. So there are ways in which there are these major upgrades in terms of using visuals, world languages, um, and other accessibility features that we just get to pull into the product the moment they happen.

Jeremy Singer: I wake up every morning, both, uh, excited about all that and terrified at the same time. Um, I spend a lot of time in College Board and personally thinking about ai, its impact on education, and really there's so much value it can add.

It is also, It can. Create a lot of issues , we're all trying to figure out how to balance that. Um, but you know, there are people who are going very deep on AI and they're coupling the use of AI [00:25:00] with the concept that. The way students learn, the way teachers teach has changed very little in the last a hundred years.

You know, your great-grandparents, my great-grandparents probably learned the same way, generally spent the most, you know, as it's happening in today's classroom. Um, they didn't have course Mojo obviously, and so then you see these people, which I like the intent. It scares me a little bit, but like this group Alpha Schools example, I don't know if you've followed, but they have a two hour, uh

traditional instruction. It's a two hour block with a student in front of a computer using ai, and then they have some hours surrounding that to do other things. And that's, they don't even involve teachers. They call 'em guides at this point. So they're really radically, uh, changing the paradigm.

Um, you know, you've obviously tried to work with the existing paradigm. Help talk about your feelings on that and, and risks and thoughts. 

Dacia Toll: Yeah. So there are two parts to the AI conversation. There's the how do we improve teaching and learning with ai, and then there's what do our young people, uh, [00:26:00] need for a future Yeah.

That will be increasingly AI powered. And I think, and we'll come back 

Jeremy Singer: to that second, in a second. Well, we have a 

Dacia Toll: tendency to have them as separate conversations, and I don't think you can. Okay. Um, I think, um, we need to, and. We're all trying to see where we think things might be headed, and there's a lot of uncertainty, but part of our belief is that, um, our human side, um, and in particular our social skills and our ability to communicate effectively with each other and work together to solve problems is gonna be even more important.

Sure. Then it has been, of course, it's problem 

Jeremy Singer: solving, communication, teamwork. And 

Dacia Toll: so part of why we don't want a silent solo experience for students with the tech is, um. We don't think that's what we have to maximize for in the future. Now, as the parent, I'm a parent of two teenage boys, um, and I'm in a constant war with them about screen time.

Like I'm actually very worried about these [00:27:00] algorithms and everything else that are maximizing for them to just uh, uh, scroll. And so we want the mojo experience to be the opposite of that. Like there's no getting away from the fact that there is, there is gonna have to be a Chromebook screen somewhere. Um, but that's why we want students talking to each other and then treating Mojo just as their handout or their third learning buddy.

And we want it to be as conversational as possible in, in all that setting. So I think that would be my concern. Um. I don't, I don't know a ton about Alpha schools, but one thing is, they do that in part as I understand it, to maximize time for project-based learning and other types of learning outside of the core academic subjects.

I think that's gesturing and the direction in which we need to go. Um, so it would be the total package ,

Jeremy Singer: . um, um. Uh, you know, my kids are now, I have 24-year-old twins, and they, um, uh, they came out okay.

Uh, probably despite me. But, I think one of the best things that I had nothing to do with is, uh,  [00:28:00] they were growing up in Maplewood, New Jersey and they were in, uh, the kindergarten class. They came home and the kindergarten, the librarian had challenged a class, uh, for what they call TV turnoff, but it's basically screen turnoff and you could go for gold, silver, bronze, and gold was.

No screen time from, you know, Monday morning until Friday evening, and they signed up for gold, like kind of just randomly. We didn't push 'em and they did that through, through middle school and so Wow. Like no screens that could be, which was, you know, which I think was much easier when they were young. It would be closer to impossible now.

But yeah, I share and, and so many of us share the concern know, the damage that screen time or social media in particular has done to kids and their attention span and so forth. And so I hear a lot of people saying We can't let aI do the equivalent of what social media did. And I'm like, you know, it's hard.

It's like to know , how to manage that. I love how you're approaching this and trying to say we're using it, but we're trying to use it in a way that builds the [00:29:00] skills they need to be successful. So I get your point of those are connected. Uh, at College Board we're spent a lot of time trying to understand, uh, what are the durable skills, what are the skills you need to be successful?

And I use an analogy, um, uh. You know, College Board for a long time. , We used to test long ago math, obviously without a calculator. Uh, we test math skills in the SAT and then. Uh, at some point we added, uh, we said, Hey, students can use a calculator. Um, not a full calculator. Some like graphing calculators, et cetera, weren't allowed, but certain types of calculator and some of the things they used to have to do without a calculator, they could do with a calculator.

And actually we had, you probably know this, but we had two sections on math, one without calculator, one with, and we were able to assess a student's math skills, even when they had access to a calculator. So I'm trying to figure out, and I'd love your thoughts on, from an AI world is there an analog here? Sometimes you want to test things where AI's turned off. You just wanna see if the student can do it. But is there a way to, like, some people are throwing their hands up and say, [00:30:00] once AI's on, you can't really assess for anything 'cause it's so powerful.

How do you think about that? 

Dacia Toll: , I hope AI is actually gonna change assessment. Um, I mean, one of the things that you mentioned earlier, these kind of benchmark assessments, in the imperfect world in which school leaders and teachers have lived, we've had to have these periodic assessments.

Yeah. Every six to 10 weeks. Um, and that's when we're, we then get all this data and try to respond. Uh, worst case on the annual state tests, um. And what AI is gonna allow us to do is to have embedded assessment on a daily basis that. Um, isn't in any way obvious that it's assessment and assessment is not even the primary job it's trying to get done.

It's trying to help student learning, but I think we can have a level of insight coming as we were describing earlier in the backend. So I absolutely do believe that the AI can be used to assess, and I know there are people working on not just assessing reading and math, but assessing, um, how [00:31:00] well students are working together.

And assessing some of these other skills. Um, so I'm optimistic about that part. 

Jeremy Singer: Yeah, me too. I'm in the middle of trying to figure out how do you assess some of these, um, durable skills and, uh. It's hard. It's hard 'cause something you could do by observation.

Some you potentially can tease out by activity. Uh, some you can assess traditionally ways of assessing. Um, but I always tell someone, you know, back in business school we did the Myers-Briggs test and if, my employment, uh, success was determined by how I turned out on the Myers-Briggs test, I could manufacture to be yes.

Whatever the mix is. And so when you tie. Uh, these lower stakes activities to high stake outcome it's hard. Um, but we're figuring it out. Well, I want to,  um, I, I asked all my guests, uh, uh, a series of rapid fire questions. Uh, um, I have four of 'em. Uh, just quick reactions. Uh, what's one education buzzword you wish we could retire?

Dacia Toll: I think personalized. Um, both because like most [00:32:00] edTech gobbledygook. I have no idea what it really means. Um, and because for the reasons we've talked about, I actually worry that's not what we should be maximizing for in this moment. Um,  , there are times when you really need an individualized pathway, but I think we need social experiences.

Yeah. That, um, were shared. I share you 

Jeremy Singer: on brand dacia, so well done. Um, what's your favorite book about education or one that deeply shaped your thinking? 

Dacia Toll: Well now I'm obsessing over effective reading instruction, so Know Better, Do Better , uh, Uh, by Meredith and David Lieben, who Yep. I Wow. Taught me a lot over the years.

I'm loving that one. And the mix of cognitive science and practical insights. 

Jeremy Singer: Yep. I love it. Uh, great one, uh, name one thing that makes you bullish and excited on the future for learners.

Dacia Toll: Oh, with in trepidation, I am gonna say ai. Um, but I think Jeremy, it sounds like you do too. Anybody who's really paying attention should be equal parts excited and, uh, nervous. There will be lots, there are already, [00:33:00] I'm afraid to say. Bad use cases of AI and education much less in society writ large. 

Jeremy Singer: Yeah.

But, 

Dacia Toll: but I think it's incumbent on us. You started about this with the educators. It's incumbent on us in a values driven way with a focus on excellent teaching to maximize the best case versions of that. 

Jeremy Singer: Yeah, and mitigate against the worst. Um, I could rant for a long time with this, but, uh, this is not my, uh, uh, uh, my right now.

Uh, and last one, what is one class, uh, you wish all students had to take? 

Dacia Toll: I would say leadership, something on leadership. Um, and I had the pleasure of visiting, um, MIT uh, just last month and a lot of their programs and labs and what I loved about their orientation is it's 50% learning in a sort of theoretical lecture reading focused way and 50% doing.

And when I think about what students are gonna need more and more, it's actually what they would get out of what might call an extracurricular activity. And that sort of [00:34:00] leadership of doing, of having to actually organize things, accomplish things, work together to solve problems. So something that focused on that with a heavy dose of emotional intelligence.

Jeremy Singer: Yeah. I love it. It's also interesting to look at colleges and how they're transforming into more, um, experiential learning, but also, um, what Northeastern has done so well, and many schools continue to copy. So, okay. We're gonna end on a positive note. You are mostly positive.

I feel like I'm, I'm bringing us down more, but, um, um, thinking three, five years out, we're looking back, um. Uh, you know, on a positive side, not the, the negative. Uh, what would you wanna see as far as, uh, how education's changed, how it's evolved, uh, and your role in that?

Dacia Toll: Well, one of the things that, you know, became equal parts overwhelming and inspiring, uh, uh, at Achievement First is we would ask, uh, students and parents what they want. And it's a long list, like Right. Um, you know, of course, you know, we care about the basics. Reading, writing, and math. Doing those increasingly well, [00:35:00] having a solid foundation, but then we want the sun, the moon, and the stars.

Like there is a whole set of additional experiences. Um, at AF we came up with this beautiful but overly complicated, uh, uh, design called Greenfield that had project based learning and. Uh, goal teams, compass learning circles, um, focused on emotional intelligence leadership opportunities. We doubled the amount of enrichment time students had and it was tough to execute.

Yeah. Um, and I was talking recently with somebody who helped us do that, and the hope is that AI can make whatever our vision is, 30 to 50% easier to actually pull off. Um, and so I, I think that's why it's so important that we start with the conversation, not of like what AI can do, but what we want. Like what is our vision for what that student experience would be like in our ideal.

And now we do have, um, this superpower to help us get closer to that [00:36:00] vision. But I worry we're having the AI conversation, not the, um. Pedagogy, student, parent, teacher experience, conversation. 

Jeremy Singer: Right. You're 

Dacia Toll: starting in the wrong place. 

Jeremy Singer: That is a, a lovely vision and frankly, I think a very doable vision.

At least the part of, as I talk to teachers and people in K 12 and college, they're already seeing a huge amount of tasks that they didn't like or they didn't particularly like grading and course planning, and that has become incredibly more efficient. And that frees up time.

And then the hard work is what you're talking about. But I love the model of like at achievement first, very ambitious stuff, but it was almost too much. But this gives you, AI could give you a way to take a significant amount off the plate. I know. That's how we're thinking of it at College Board is this idea of if we can make certain tasks that are less, um, needed or, or less need human time and allow our staff to focus on bigger opportunities. Great. I also like how you started to talk a little about Achievement First. I do wanna keep that option to come [00:37:00] back and talk to you more 'cause it's such a great example of success. Last thing I'll say is if, uh, teachers and districts listening, uh, want to get engaged with Course Mojo, what should they do? 

Dacia Toll: Uh, they could go to our website and there's a way to just find us there. They could email us at [email protected]. Um, and I'm always happy to do a low stakes demo for folks because I think.

Um, you need to see it, you need to actually experience it and not, um, just rely on what,, what can sound like all the same buzzwords. 

Jeremy Singer: having seen it, it's, uh, awesome and a big advocate of it. So thank you so much, Dacia. I appreciate you spending the time today, uh, talking about the great work you're doing.

Dacia Toll: Thanks, Jeremy. Great to see you too. 

Jeremy Singer: Awesome.