Dacia Toll | Coursemojo - Transcript
Jeremy Singer: [00:00:00] I’m Jeremy Singer, president of College Board, and this is the Education Equation. I’ve spent my career grappling with what truly drives student success. On this podcast, I’ll talk with people who are researching, building, and scaling solutions that matter. Every episode will go beyond the hype and focus on data and evidence to see what’s actually working.
Let’s stop guessing, and let’s figure out what works.
Today’s guest is Dacia Toll, co-founder and co-CEO of Coursemojo, as well as the co-founder of Achievement First, a charter network of nearly 40 schools. We dig in to two urgent issues: eighth-grade reading scores, which Dacia calls a national emergency, and teacher retention. We explore the role of AI, how Coursemojo supports student learning, gives teachers real-time insights, and [00:01:00] frees up time for deeper, more human instruction. It’s a clear-eyed, hopeful conversation about using technology wisely to tackle some of the toughest challenges in schools. Let’s dive in.
Dacia, it’s a bit crazy to think we met over a quarter century ago, and it’s been thrilling for me to watch you as you built one of the most impactful charter networks in Achievement First, and now we’re going in a very different direction, but also very impactful, with Coursemojo. There’s a ton we can discuss, but let’s agree today we’re going to focus on Coursemojo. I’ll keep an option to dig into Achievement First another time.
So, with that, let’s jump in. Help our listeners understand why you founded Coursemojo and what specifically it’s designed to address.
Dacia Toll: We are going after two big challenges with our school partners. The first is reading achievement, which we all saw on the very sobering NAEP, or national reading data. It’s at a [00:02:00] 30-year low for where eighth graders are as they head off to high school. And so, while we know students took a big step backward during the pandemic, frankly the decline was happening long before that. And it’s now … especially the number of students that are scoring below basic … it’s a national emergency …
Jeremy Singer: No question.
Dacia Toll: … in terms of the implications for young people in the rest of their lives.
So, that’s the first big one. The second one, and it’s related, is teacher retention. We have got to figure out a way to make this a profession that more people want to join and stay in, and get ever better at. And so those are the two biggies.
Jeremy Singer: That’s awesome. Two incredibly important challenges facing all of us and all of education today.
Why don’t you give a quick sense of like how Coursemojo works and how it addresses those two pieces?
Dacia Toll: One of the biggest differences between what we’re doing and what so many others are doing is we are not a supplemental. The vast majority of EdTech products are [00:03:00] something extra that teachers have to squeeze in somehow for 20 minutes a day or twice a week. And we are trying to make the core instruction that teachers are already trying to do for the day, the ELA block, the English Language Arts block, we’re trying to make that more effective, more engaging, and easier. And so, we are aligning to the high-quality instructional materials that a lot of districts and schools have adopted. What we’re doing is embedding the AI supports for students and teachers in those materials.
Jeremy Singer: That’s great. And I want to jump into the specifics, but I just want to also underscore, when you’re designing a product, you always want to understand your users. What’s frustrating … In my position, I see so many startups that don’t talk to teachers or don’t really understand what teachers need, and they develop these supplemental products. And I think both of us know, you talk to the vast majority of teachers, the last thing they want is something else that they have to fit into their [00:04:00] schedule, their school day.
They are already overwhelmed with what they have to do. And so, I love the fact that you’ve really pinpointed and anchored in on trying to provide something that helps them do what they already do more efficiently. And I’m sure that helps with the teacher reception. But go on. Tell us more about how it works.
Dacia Toll: Well, what that looks like then … there’s both student-facing supports and teacher-facing supports, and it all works together. So, what that looks like from a student perspective, and this gets back to another big difference, is the vast majority of EdTech tools are also silent solo experiences. And so, kids put on their headphones, and they go down their own personalized pathway.
That’s not this. We actually want students working together in partners with Mojo. Mojo’s the AI-powered teaching assistant. So, it’s like you have a third learning buddy in your group. It’s often partners, sometimes more than that, and students … Because we’re anchored in the curriculum, it’s the same rich, wonderful, complex [00:05:00] text that they were already reading that day, the same thinking questions that they were already grappling with, but now, as a student said, it “It’s like the handout is talking to me.”
Jeremy Singer: I love that.
Jeremy Singer: So, they talk to each other, and they talk to Mojo, and they go from … Mojo categorizes student learning, but sometimes they start it confused, but trying, so it’s not an effortless issue.
Right. But they’re not fully getting the deeper meaning, the metaphorical meaning of a poem or the author’s craft of a novel or the big ideas of a nonfiction. Then Mojo nudges them, and they support each other to go to partial understanding and then to full understanding.
And so, while that’s happening, while students are working together, teachers have a live dashboard that shows them every student’s level of understanding for every single question. Many have described that as a game changer. If you’ve got a room of 25 to 30 middle school students, and you can [00:06:00] quickly see who needs your help and what they need your help with, it just helps you to be so much more strategic with your conferencing.
Then after students work together for 10 to 15 minutes, the teacher will often pause the class. And then go to their … to Mojo. And with a click of a button, it will tell you the two most common misconceptions in your class right now. Teachers review those, and Mojo suggests potential discussion questions. Teachers make the choice, but now they’re doing it in this wonderfully data-informed way.
Jeremy Singer: Yeah, it’s great. Twenty plus years ago when David Coleman and I started Grow Network, a lot of the same concepts was trying to take data, this was from the summative exams, and then feed it back to the classroom teacher so it could actually inform instruction. It was very rudimentary compared to what you’re doing.
I think one of the challenges we had … and I’d love to hear what your experience has been … is we continually overestimated the complexity of the data. We provided a lot of data, and it was just too much. For a lot of teachers, again, who have so many … it’s not that they’re not facile with data, it’s just they have so much stuff they’re trying to do.
So, [00:07:00] we had, I think, an overly ambitious idea of how they could apply the data. Over our time, we just kept simplifying, simplifying, and narrowing. But what have you found, and how do you deal with that challenge?
Dacia Toll: So, I think the big game changer for when you and many others were working on these same sort of problems and opportunities that now is AI.
AI can do a number of things. First, it frees us from a world of multiple choice. So, the kinds of questions we’re able to ask, the kinds of student responses, we’re able to go after is … we can now ask students what we want to ask. Almost all classroom questions are a wonderfully open-ended question.
Right. Right. And the AI can evaluate those responses. The second thing is it can analyze large amounts of unstructured data almost instantly …
Jeremy Singer: Yeah.
Dacia Toll: … and so you always had I think after-the-fact part of it. [00:08:00] So, what we are trying to do is, in this moment, what is the teacher trying to do?
When I was at Achievement first, we used to call it rapid feedback. Different places call it different things, but when students are working, you, as a teacher, are circulating around the classroom and trying to both support individual students and then figure out what’s the trend, and student misconception. It’s wonderful and it’s hard to do, particularly in a class where you’ve got … sometimes unfortunately management concerns have to be the biggest concern.
So, the good news is Mojo’s so engaging that students are working effectively with their partners. But then it really tells you, so it’s in that moment: What’s the information I need to take? What’s the action I want to take? Yeah, so it’s very simple colored dots that a teacher drew for us in one of our earliest pilots.
Back to your point about the importance of teacher feedback, and then it’s the top two misconceptions. We could give you seven. We give you two.
Jeremy Singer: Right.
Dacia Toll: [00:09:00] A suggested discussion question for each, and they’re able to make that real-time choice about what would be best.
Jeremy Singer: I love it. Now I’m thinking back to business school. We studied just-in-time manufacturing, and this is just-in-time instruction, which is brilliant.
And that was a model we couldn’t do then, after the fact. So, I think the timeliness makes a ton of sense in driving success. That is awesome. What has been the teacher response? I mean, has everybody embraced it?
Have you got challenges? How has that worked?
Dacia Toll: The overall response has been very positive. I mean, we have gotten critical, constructive feedback, and that’s made us so much better. But we survey teachers regularly. We talk to them informally all the time. In our surveys last year, the average teacher rating across a number of dimensions was a 4.2 out of 5 …
Jeremy Singer: That’s awesome.
Dacia Toll: … which for our first full year was very strong, not just in terms of how much they think it’s supporting student learning, but to that second goal we have, how much [00:10:00] it’s making their jobs easier and more enjoyable, how much it’s helping them implement the core curriculum. That’s been very positive.
As I’ve said, it’s every step of the way. It’s teachers and students who’ve said to us, “Hey, this is good, but could you do this? Or this thing’s a little frustrating” or “It takes me a little too long” or “ I think there’s a better way.” Those have been the breadcrumbs that we just continue to follow to make this even better.
Jeremy Singer: Yeah. It’s awesome. And I feel that your instructional experience and being in the classroom, developing schools and being a school leader, really allows you to do things very differently. And I am always nervous. I mean, there’s a lot of successful people coming in from outside, but that experience is critical. I also love how you’re bringing up the 92%. One of the things that I appreciate around your approach is … and we’re … I think we have a good kinship … is measuring and using data to evaluate impact. So, I know there was recently some good data that came to you about the impact.
[00:11:00] Can you give us a quick summary of what it was and the results?
Dacia Toll: Yeah, impact is always most important. It is interesting. I find myself being fairly critical of the education technology industry. I’ve been a skeptic for a long time. I think it all sounds good on the front end, but routinely people overpromise and underdeliver.
Jeremy Singer: Yeah. And you were a recipient. I mean, running Achievement First, you were obviously trying different things and so you saw firsthand that a lot of the promise didn’t actually turn into reality when you adopted it.
Dacia Toll: Yeah. And so, we were holding our breath for ultimately state test scores. Because you see progress along the way, but sometimes it doesn’t translate, and it has translated substantially. Two of our biggest partners have gotten their results back. Aldine ISD in Texas. They had the largest reading gains of any large district, of which they’re like 115 in the state of Texas. More than 10,000 kids in the state.
So, they really blew it out. Now part of that is credit to them. They had done some really thoughtful work on coaching. [00:12:00] They’d adopted HQIM, but then there was a distinct Mojo layer. So, they piloted Mojo in some middle schools and not others. And the schools that piloted Mojo on top of the districtwide gains had another six percentage points.
Jeremy Singer: Wow.
Dacia Toll: … which may not sound like a lot ….
Jeremy Singer: It does sound like a lot to me.
Dacia Toll: … but for middle school reading achievement, that makes you the most improved in the state. It is hard to move because reading, as you know, is so complex. And then Sumner County in Tennessee. Similarly, they had a dozen middle schools.
They had six use Mojo six, six not. The schools and teachers and students who did had an eight-percentage-point gain.
Jeremy Singer: Wow. Congratulations. Yeah, that’s awesome. Thank you.
Dacia Toll: We’re doing a variety of research studies, and they have told us that that meets the ESSA Tier 2 standard for evidence of impact, which in our first year, we’re certainly excited about.
Jeremy Singer: That is awesome. You know, we talk a lot about … I think maybe when you mentioned your frustration with innovation and, really, interventions [00:13:00] or programs that change student outcomes. And there’s again, so much hype but so few results, which is part of why I started this podcast. But, you know, part of it’s a challenge, right?
It’s hard to do a randomized control trial in a school. There’s just so many factors that you can’t control. So, to have a test and a control environment is hard. But I love it, and while it’s not perfect, I love that you have in both these districts a set of schools where Coursemojo was being used and then a set of schools that weren’t, And you can see the differential gain. So, you should be super proud of that and, I’m sure, excited. So, take us through what’s next in the evolution. You’ve got something out. It’s already having some impact. How do you see it expanding? How do you see it growing?
Dacia Toll: Well, we’re certainly adding more grades, more curricula that we support. So, we were aligned to three curricula last year: Wit and Wisdom, EL, and My Perspectives, and we’ve now expanded that to a list of seven. Then we’ve gone from one grade to many grades, focusing really though on that middle school [00:14:00] challenge where, after the work, the very important work on phonics and fluency … sort of the bottom half of Scarborough’s Rope and the Science of Reading … kids hit what’s been called a comprehension wall. Making that next jump up to really being able to think critically about the text and understand increasingly complex ideas.
It’s a thinking job as much as a reading job. So, that’s the part that we are focused on. So, we’ve done that. Then in terms of the product itself, we maximized first for the everyday classroom experience of students and teachers and making sure that we were adding value in that moment, helping teachers not just save time but be more effective.
Have more teachers able to execute the moves of effective teachers. And now we’re shifting to expanding back to the power of AI and the ability to analyze student work almost instantly. It doesn’t just have to be today’s work. It can be over time, which will enable us to see [00:15:00] trends in student learning over time. Then we are adding coach tools as well. So beyond just the individual teacher, how can school leaders and coaches have visibility into that student learning and teacher practice?
Jeremy Singer: It’s awesome. Really inspiring. I also love so many parts of this, but that you're focusing on middle school.
You laid out this sort of literacy development so well, but the other thing that is a challenge is a lot of people have given up on any progress in middle school. You know, there’s elementary and then there’s high school, and people think this is just a way station.
So, I love that that is one of the hardest places to tackle. From my perspective at College Board, what’s interesting is we ... in AP Exams, one of the best parts of the exams are the document-based questions, the DBQs, which really force a student to analyze multiple sources.
And these can be texts. They can be pictures. They can be letters, and then synthesize the meaning. And I’ve always thought if we could bring [00:16:00 DBQ into middle school, that would be another angle exactly aligned with that.
I want to go back to something you said because a lot of the listeners, as all of us, are very interested in AI today. And you mentioned two great benefits of AI, which I don’t think everybody thinks about. But one is that you can go well beyond multiple choice. You can start to do longer writing work, which is great. And then I also love the idea of its ability to analyze vast amounts of data.
We’re just at the tip of the iceberg on that. But on the first part of analyzing writing and giving writing feedback, tell me how you do it. Do you train a model on these prompts? Have you got to sophistication where it doesn’t need training?
Like, just help our listeners understand literally how that works.
Dacia Toll: Yeah, one of the things that we’re doing differently, that frankly takes a lot of time, is that for every single question, we] are telling the AI what the levels of understanding are, what constitutes criteria for success, what constitutes level 101 understanding the [00:17:00] variety of answers that would do that. A level 201 …
Jeremy Singer: You have some sort of rubric, but is it specific to each text?
Dacia Toll: It’s specific to each text, and it’s specific to each question.
Jeremy Singer: Oh, wow. So. it’s a lot of work. Yeah.
Dacia Toll: I think part of it is a lot of tools out there are these generic AI bots that maybe have been tuned and trained, but I think when you go beneath most of them, it’s just prompt engineering.
AI is powerful, and it will give you an acceptable answer some amount of time. Some amount of time, it will give you an incorrect or unacceptable answer. And when you’re working with struggling readers, our tolerance for answers that are not accurate is pretty low.
Jeremy Singer: Yeah, and I don’t think a lot of people understand that we’re … in many cases, if there’s some random wrong answer, it’s okay. When you’re doing a search online or you’re doing something, but in education, it can be quite impactful.
Dacia Toll: And it’s not, I think the large language models themselves are [00:18:00] wired to be our assistant and to tell us the answers with a lot of confidence, and they are working on increasing …
Jeremy Singer: …like a few people we know but …
Dacia Toll: That’s not what we need in an educational setting. We need a sophisticated tutor. And that’s not giving you the answer, that’s asking you to just write the questions, and that doesn’t drain the rigor and the thinking out of it.
These texts are nuanced and what the level of understanding is. So now we are increasingly using AI and human authors to author the criteria I’m describing to you. So, the texts and many of the questions, the initial questions themselves, come from this core curriculum. But what we are using the AI to do is to develop all of the criteria for each question.
Then the scaffolded questions, and then the transferrable takeaway. Because the other thing is we get to read this wonderful text today, and [00:19:00] we get to spend time grappling with it and getting to deeper levels of understanding in a way that really can be fun. Then we need to understand what we did today that’s transferrable as effective readers and writers.
That’s the way in which we’re using the AI. It’s to categorize that open-ended response. And as you said, we can do longer form writing, but the other big change is it’s almost like we can have a conversation with the AI. In the initial part of the activity, there’s a lot of fast passing between the students and each other and the AI, and it feels like we’re just having an informal conversation in which we’re grappling with what this text is trying to tell us and why, and how is the author going about it? And then we have a class discussion with the full group, and then we come back, and often it ends with a writing piece.
Jeremy Singer: Right. And you’ve talked about, we’ve all talked about, this Socratic method that was once very much in style in real life. It fell out of favor for a myriad of reasons, but seems like it’s [00:20:00] having another run now with AI because AI can be this partner in that Socratic discussion.
You’re using AI. It’s literally helping the student along in their thinking.
Dacia Toll: Right. Yeah, I could talk about … get in the weeds and talk about all the specifics all day long, Jeremy, so help me here. But when I think about what’s driving the results, the biggest thing is kids are doing more thinking, more reading, and more writing.
Jeremy Singer: Right.
Dacia Toll: And time on task. Because we’ve started with high-quality instructional materials. If we weren’t starting there, this would be a different conversation, but because we’re starting with text worth reading and questions worth asking, and because Mojo’s favorite move is actually to direct students back to a particular part of text ...
Jeremy Singer: Yeah.
Dacia Toll: … to a particular paragraph, a particular sentence. And get them to keep doing that thinking. Yeah. And because kids like Mojo, and he, when they send them a message, he’ll give them a little emoji, thumbs up, a dance guy. There’s an ever so slight gamification mostly around praise for [00:21:00] authentic student work.
Jeremy Singer: Yeah..
Dacia Toll: But it’s very subtle. I don’t believe in dialing that up too much, but the kids like it, and so they like the overall experience. One of the things we’re proud of as teachers to say: They’re more likely to do partner and small-group work in class. Because they now have eyes everywhere with Mojo.
So, it’s more social. Actually more discourse. More motivating.
Jeremy Singer: Yeah. Yeah.
Dacia Toll: And so, then the kids are putting forth more effort. Because they’re receiving up to three rounds of feedback on every question, exponentially more feedback than would be possible otherwise, that effort is increasingly effective.
Jeremy Singer: What’s so exciting is you have a very intentional, instructional, pedagogical approach, but you’ve also thought about the engagement emotional side. I’ve lived at places where we’ve had great pedagogy, but it couldn’t engage users, or we really can engage users, but the pedagogy is weak, and so you seem to be balancing that line.
[00:22:00] I agree with you. I think gamification is another area that’s been way overstated. But at times it can work. I always worry it’s like a fad. When my kids were young, they used to do the streaks on some of the apps where they’d post something every day and have friends send a message.
I don’t even know what it was … probably a bad parent, but I watch Duolingo and my kids are older, but I see younger kids who want to keep streaks alive or do something, and they’ve really gamified. I don’t know if that will stick. Those trends, if you look, Snapchat, there were streaks in Snapchat, and then it disappeared. If you look at the presence on Google, some of those things, I do think over-alignment with that is questionable. I do want to do one more question.
Then I want to move on to something else. But there’s a lot of scaffolding you’re providing to the AI to help the student through the work. The speed at which AI is improving … I mean, it’s mind boggling.
Are you thinking there’s a world where you’re not going to have to … where the AI will get to a point in your confidence where the amount of work to prep a particular question or particular prompt … the amount of pre-work you’ll have to do will be reduced.
Dacia Toll: [00:23:00] I hope so.
It is getting a lot faster already because we’re using AI to help us set the criteria for other AI to use, to evaluate. So yeah, I'm certainly optimistic that that will help. I think there are a number of different solutions that layer the AI onto itself to do some self-checks. We also now have … before any activity sees live students, we have a whole bunch of AI students basically push on it and test on it. And optimize prior to that. And then of course, once we go live with students, they give us good feedback too. I think there are a lot of ways in which it will become more efficient.
I am skeptical that we will reach a truly strong pedagogical AI on its own …
Jeremy Singer: Yeah.
Dacia Toll: … because I don’t think it’s being built for that use case. I think it’s being repurposed from this, [00:24:00] as we described, more assistant expert use case. So, the ability to make a key point about the difference between an inference question and a right there. I think there’s just a number of ways. Yeah, I would love to be pleasantly surprised, but our assumption is that, to your point, you have to really dig in and maximize for strong teaching. Strong teaching support …
Jeremy Singer: Right. Right.
Dacia Toll: … for the teachers and for students.
Jeremy Singer: I think I’m a little more bullish.
I agree with the point. It’s not designed for some of these things, but I’ve watched AI, using it for things that definitely weren’t designed for being increasingly effective. So, look, you said it’d be a pleasant surprise. Let’s hope you get that surprise.
Dacia Toll: One thing I will say Jeremy, to your point, it just does get so much better. So much faster.
Jeremy Singer: Yeah.
Dacia Toll: We haven’t talked aboutthis yet, but one of the things that’s possible is real-time translations, not of the entire experience, but in a targeted way to help students understand a [00:25:00] particular question, a particular piece of feedback.
It can, of course, translate the whole experience. That’s all AI. People are blown away when there’s the list of 65 different home languages. That’s the AI. And speech to text. And text to speech. Yeah. So, there are ways in which there are these major upgrades in terms of using visuals, world languages, and other accessibility features that we just get to pull into the product the moment they happen.
Jeremy Singer: I wake up every morning, both excited about all that and terrified at the same time. I spend a lot of time in College Board and personally thinking about AI, its impact on education, and really there’s so much value it can add.
It is also … It can create a lot of issues. We’re all trying to figure out how to balance that. But you know, there are people who are going very deep on AI, and they’re coupling the use of AI with the concept that the way students learn, the way teachers teach, has changed very little in the last hundred years.
Your great-grandparents, my great-grandparents probably learned the same way. [00:26:00] They generally spent the most … as it’s happening in today’s classroom. They did n’t have Coursemojo obviously, and so then you see these people, which I like the intent. It scares me a little bit, but like this group Alpha Schools example, I don’t know if you’ve followed, but they have a two-hour, traditional instruction. It’s a two-hour block with a student in front of a computer using AI, and then they have some hours surrounding that to do other things. And that’s … they don’t even involve teachers. They call them guides at this point. So, they’re really radically changing the paradigm.
You’ve obviously tried to work with the existing paradigm. Help us talk about your feelings on that, and risks and thoughts.
Dacia Toll: Yeah. So, there are two parts to the AI conversation. There’s “How do we improve teaching and learning with AI?” Then there’s “What do our young people need for a future that will be increasingly AI powered?”
Jeremy Singer: And we'll come back to that second.
Dacia Toll: Well, we have a tendency to have them as separate conversations, and I don’t think you can. [00:27:00]
Jeremy Singer: Okay.
Dacia Toll: I think we need to. We’re all trying to see where we think things might be headed, and there’s a lot of uncertainty, but part of our belief is that our human side, and in particular our social skills and our ability to communicate effectively with each other and work together to solve problems, is going to be even more important than it has been.
Jeremy Singer: Sure. Problem-solving, communication, teamwork. Yeah.
Dacia Toll: Part of why we don’t want a silent solo experience for students with the tech is we don’t think that’s what we have to maximize for in the future. Now, as parent, I’m a parent of two teenage boys, and I’m in a constant war with them about screen time.
I’m actually very worried about these algorithms and everything else that are maximizing for them to just scroll. And so, we want the [00:28:00] Mojo experience to be the opposite of that. There’s no getting away from the fact that there is … there is going to have to be a Chromebook screen somewhere, but that’s why we want students talking to each other and then treating Mojo just as their handout or their third learning buddy, and we want it to be as conversational as possible in that setting. So, I think that would be my concern.
I don’t know a ton about Alpha schools, but one thing is they do that in part as I understand it is to maximize time for project based learning and other types of learning outside of the core academic subjects.
I think that’s gesturing in the direction in which we need to go. It would be the total package.
Jeremy Singer: My kids are now … I have 24-year-old twins, and they came out okay, probably despite me. But I think one of the best things that I had nothing to do with is …. They were growing up in Maplewood, New Jersey, and they were in the kindergarten class. They came home, and the kindergarten librarian had challenged a class for what they call [00:29:00] TV turnoff, but it’s basically screen turnoff, and you could go for gold, silver, bronze, and gold was.
No screen time from Monday morning until Friday evening, and they signed up for gold, just randomly. We didn’t push them, and they did that through middle school. No screens that could be … which I think was much easier when they were young. It would be closer to impossible now.
But yeah, I share, and so many of us share, the concern the damage that screen time or social media in particular has done to kids and their attention span and so forth. And so, I hear a lot of people saying, “We can’t let AI do the equivalent of what social media did. And I think it’s hard to know how to manage that. I love how you’re approaching this and trying to say we’re using it, but we’re trying to use it in a way that builds the skills they need to be successful. So, I get your point of those are connected. At College Board, we’re spending a lot of time trying to understand: What are the durable skills? What are the skills you need to be successful? [00:30:00]
And I use an analogy. College Board, for a long time … We used to test long ago math obviously without a calculator. We test math skills in the SAT, and then at some point we added … we said, “Hey, students can use a calculator.” Not a full calculator. Some graphing calculators, etc. weren't allowed, but certain types of calculator and some of the things they used to have to do without a calculator, they could do with a calculator.
And actually, we had, you probably know this, but we had two sections on math, one without calculator, one with calculator, and we were able to assess a student’s math skills, even when they had access to a calculator. So, I’m trying to figure out, and I’d love your thoughts on from an AI world, is there an analog here? Sometimes you want to test things where AI’s turned off. You just want to see if the student can do it. But is there a way to … some people are throwing their hands up and saying, “Once AI’s on, you can’t really assess for anything because AI’s so powerful.
How do you think about that?
Dacia Toll: I hope AI is actually going to change assessment. [00:31:00] I mean, one of the things that you mentioned earlier, these kind of benchmark assessments, in the imperfect world in which school leaders and teachers have lived, we’ve had to have these periodic assessments every six to 10 weeks. That’s when we’re … we then get all this data and try to respond. Worst case on the annual state tests. And what AI is going to allow us to do is to have embedded assessment on a daily basis that isn’t in any way obvious that it’s assessment, and assessment is not even the primary job it’s trying to get done.
It’s trying to help student learning, but I think we can have a level of insight coming as we were describing earlier in the backend. So, I absolutely do believe that the AI can be used to assess, and I know there are people working on not just assessing reading and math, but assessing how well students are working together.
And assessing some of these other skills. I’m optimistic about that part.
Jeremy Singer: Yeah, me too. [00:32:00] I’m in the middle of trying to figure out how do you assess some of these durable skills. It's hard. It’s hard because it’s something you could do by observation.
Some you potentially can tease out by activity. Some you can assess traditionally ways of assessing, but I always tell someone … back in business school we did the Myers-Briggs test, and if my employment success was determined by how I turned out on the Myers-Briggs test, I could manufacture whatever the mix is. And so, when you tie these lower-stake activities to high-stake outcome, it’s hard, but we’re figuring it out.
Well, I want to … I ask all my guests a series of rapid-fire questions. I have four of them. Just quick reactions. What’s one education buzzword you wish we could retire?
Dacia Toll: I think personalized because, like most EdTech gobbledygook, I have no idea what it really means and because for the reasons we’ve talked about, I actually worry that’s not what [00:33:00] we should be maximizing for in this moment. There are times when you really need an individualized pathway, but I think we need social experiences that were shared.
Jeremy Singer: On brand, Dacia. So well done. What’s your favorite book about education or one that deeply shaped your thinking?
Dacia Toll: Well now I’m obsessing over effective reading instruction, so Know Better, Do Better: Comprehension by David and Meredith Liben, who taught me a lot over the years.
I’m loving that one. And the mix of cognitive science and practical insights.
Jeremy Singer: Yep. I love it. Great one. Name one thing that makes you bullish and excited on the future for learners.
Dacia Toll: Oh, with trepidation, I am going to say AI. I think, Jeremy, it sounds like you do too. Anybody who’s really paying attention should be equal parts excited and nervous. There will be lots, there are already, I’m afraid to say, bad use cases of AI and education, much less in society, writ large.
Jeremy Singer: Yeah.
Dacia Toll: But I think it’s incumbent on us. You started about this with the educators. [00:34:00] It’s incumbent on us in a values-driven way with a focus on excellent teaching to maximize the best case versions of that.
Jeremy Singer: Yeah, and mitigate against the worst. I could rant for a long time with this, but this is not my slot right now. Last one: What is one class you wish all students had to take?
Dacia Toll: I would say leadership, something on leadership. I had the pleasure of visiting MIT just last month and a lot of their programs and labs. What I loved about their orientation is it’s 50% learning in a theoretical lecture reading focused way and 50% doing.
And when I think about what students are going to need more and more, it’s actually what they would get out of what might called an extracurricular activity. And that leadership of doing, of having to actually organize things, accomplish things, work together to solve problems. So, something that focused on that with a heavy dose of emotional intelligence. [00:35:00]
Jeremy Singer: Yeah. I love it. It’s also interesting to look at colleges and how they’re transforming into more experiential learning, but also what Northeastern has done so well, and many schools continue to copy.
So, okay. We’re going to end on a positive note. You are mostly positive.
I feel like I’m bringing us down more, but thinking three, five years out, we’re looking back, on a positive side, not the negative. What would you want to see as far as how education’s changed, how it’s evolved, and your role in that?
Dacia Toll: Well, one of the things that, you know, became equal parts overwhelming and inspiring at Achievement First is we would ask students and parents what they want. And it’s a long list. We care about the basics. Reading, writing, and math. Doing those increasingly well, having a solid foundation, but then we want the sun, the moon, and the stars.
There is a whole set of additional experiences. At AF we came up with this [00:36:00] beautiful but overly complicated design called Greenfield that had project based learning and goal teams, compass learning circles focused on emotional intelligence leadership opportunities. We doubled the amount of enrichment time students had, and it was tough to execute.
I was talking recently with somebody who helped us do that, and the hope is that AI can make whatever our vision is, 30% to 50% easier to actually pull off. I think that’s why it’s so important that we start with the conversation, not of what AI can do, but what we want. What our vision is for what that student experience would be like in our ideal.
And now we do have this superpower to help us get closer to that vision. But I worry we’re having the AI conversation, not the pedagogy, student, parent, teacher and experience, conversation.
Jeremy Singer: Right.
Dacia Toll: Starting in the wrong place. [00:37:00]
Jeremy Singer: That is a lovely vision, and frankly, I think a very doable vision.
At least the part of … as I talk to teachers and people in K−12 and college, they’re already seeing a huge amount of tasks that they didn’t like or they didn’t particularly like, such as grading and course planning, and that has become incredibly more efficient. And that frees up time and then the hard work that you’re talking about. But I love the model at Achievement First. Very ambitious stuff, but it was almost too much. But this gives you … AI could give you a way to take a significant amount off the plate. That’s how we’re thinking of it at College Board. This idea that if we can make certain tasks that are less needed or have less need for human time and allow our staff to focus on bigger opportunities.
I also like how you started to talk a little about Achievement First. I do want to keep that option to come back and talk to you more because it’s such a great example of success. Last thing I’ll ask is if teachers and districts listening want to get engaged with Coursemojo, what should they do?
Dacia Toll: They could go to our website, and there’s a way to just find us there. They could email us at [email protected]. I’m always happy to do a low-stakes demo for folks because I think you need to see it. You need to actually experience it and not just rely on what can sound like all the same buzzwords.
Jeremy Singer: Having seen it, it’s awesome. I’m a big advocate of it. So, thank you so much, Dacia. I appreciate you spending the time today talking about the great work you’re doing.
Dacia Toll: Thanks, Jeremy. Great to see you too.
Jeremy Singer: Awesome.
Thanks for tuning in today. Join the conversation by following the Education Equation wherever you listen to podcasts.