Listen in your favorite podcast provider
Tatyana Vdovina: [00:00:00] Any assessment can be a formative assessment or a summative assessment, right? So the assessment itself, that doesn’t really matter because if we have a quiz that we’ve been teaching for three days about a topic and then we have a quiz that we give to the students, let’s say it’s multiple choice. We give it to them, we grade it, and then we forget about that, right?
And maybe we take the grade and that contributes like 5 percent of the final quarter grade or marking period grade. That’s a summative assessment. You maybe go over the results with the class and you incorporate it in your teaching and students learning. That’s what makes an assessment a formative assessment.
So it can be the exact same assessment. But what we do with the result of the assessment will determine whether it’s formative or summative.
Justin Hewett: Hey, everybody. Welcome to the ML chat podcast. We’re glad you’re here. My name is Justin Hewett. I am your host today. Joined by my co host, Miss Mandi Morris over [00:01:00] here.
What a delightful conversation we just got to have, Mandi.
Mandi Morris: Absolutely. Tatyana was a wonderful guest. She brings years of experience in ML education and assessment and just a, an incredible conversation with her.
Justin Hewett: Oh, it really was. Tatyana Vdovina is from Moldova. She came over when she was 13. She was a newcomer English learner.
She had spoken Russian. She came in and you’ve got to listen for her experience, but she got dropped off at the wrong school on her very first day. And you can imagine how that went. And she talks a little bit about that. And I think that ultimately it was good to relate with because it informed a lot of what she ended up doing in her career.
And even today, what she’s doing when building tools and kits and fact sheets and really serving multilingual learners across the nation, as she works at the National Clearinghouse for English Language Acquisition, serving Oella.
Mandi Morris: [00:02:00] Absolutely. And it’s interesting to hear about how she was only in ELD for about a year and a half before they just put her in gen ed classes.
And now as an adult and in her adult profession has this passion for assessment, again, really informed by her own experiences. So this is a great episode to listen to today.
Justin Hewett: Yeah. There was no assessment to determine whether she should stay receiving certain language services or not. And so it is, that is an interesting connection.
I love her perspective on assessment. She taught fundamental or foundational assessment classes at the college university for 15 years. She has a great perspective on it that I think is really helpful, especially in thinking about the relation of formative assessment and summative assessment. And so I think you’re going to really enjoy this conversation today with Tatyana.
Let’s go ahead and go jump in. Tatyana is the deputy project director for the National Clearinghouse for English Language Acquisition. She has spent the last 20 years in the field of second language acquisition, applied [00:03:00] linguistics, language assessment, and professional development. Tatyana received a bachelor’s of arts from Indiana University in history and her master’s from the University of Maryland in second language acquisition.
Tatyana, we are thrilled to have you join us here on the ML chat podcast. Welcome.
Tatyana Vdovina: Oh, thank you so much. I’m so glad to be here.
Justin Hewett: Oh, you bet. Our pleasure. It’s we’re really looking forward to this conversation and looking forward to getting to know you better. You have a really fun story that we’d love to unpack and learn.
And then we’d love to dive into also the great work that you’re doing today. Maybe to get started, tell us a little bit about the work that you do today, if you don’t mind, just so that we can start with the end in mind.
Tatyana Vdovina: Oh yeah, absolutely. Like you said, I’m the deputy project director for the National Clearinghouse for English Language Acquisition Program.
I work for the, uh, consulting company called Manhattan Strategy Group. We’re based in Bethesda, Maryland. And we serve in my, on my [00:04:00] project, we serve the Office of English Language Acquisition at the Department of Education. So I’ve been with this project for five years, actually. It was recently my work anniversary and I worked on them Stella, which is the acronyms.
for the National Clearinghouse for English Language Acquisition for that and for this entire time. And I don’t know how much you guys know about Ancela, but it’s actually a really great resource in terms of supporting education, educators of English learners and multilingual learners themselves, their families, researchers in the field.
We have a website that we generate a lot of products for educators like. Things like webinars and fact sheets on various demographic points and achievement data and all kinds of stuff. We have podcasts, we publish briefs, we do a biannual report to Congress on the state of English language education and English learners in the [00:05:00] U.
S. So just all kinds of stuff. I. In my role as a deputy PD, I dabble in almost all of it. I work a lot on organizing webinars, working with SMEs, proposing different topics. I also work on fact sheets and different infographics about English learners. I author them as well as coordinate with different experts in the field on putting them together.
Yeah. So we do little information. They’re called information elevated data visualization videos. I record podcasts on the other side of the podcast, usually with different various experts from the field on various topics that touch upon education of multilingual learners. It’s a great resource. If you haven’t seen the Ancela website, definitely check it out.
If you just put N C E L A into your search engine, it will come up.
Justin Hewett: Oh, I love it. Fantastic. And Smee, for those of you that are getting more [00:06:00] familiar with some of these acronyms, is subject matter experts that Tatyana is interviewing and working with. Sounds like an amazing resource, and we’re excited to, to learn more and talk about some of those tools and resources.
That can be available.
Mandi Morris: Tatyana, it’s so interesting learning about all the work that you’re doing now in your role. It would be wonderful to go back and tell us what brought you to ML education. What drove you to this section of education? Go back and tell us a little bit about the beginning for you.
Tatyana Vdovina: Oh, sure. Yeah, absolutely. I think when To be honest, I feel like I ended up in this field by accident almost, but I think it worked out really well and I feel like it really connected with my background as an English learner myself. So I came to the U S when I was 13 years old in 1993, I was in seventh grade.
We, me and my family moved from Moldova. Which at that point was an independent [00:07:00] country, but historically, when I was growing up, it was part of the Soviet Union, dating myself here, but in 1991, the Soviet Union broke up and Moldova gained its independence. So I grew up there. I am a Russian speaker. I went to a Russian speaking school there.
And when I moved to the US, I didn’t know any English. So I was one of those newcomer students who we now support, who came to the classroom and didn’t speak pretty much a word of English. On my first day of school in South Bend, Indiana, which is where we moved, I, we only had an ESL program in one middle school, one high school and one elementary school.
The middle school that I had to go to was actually out of the district where I lived. So I had to, I remember like somehow it was arranged that I would take like a taxi to go to school. And on the first day of school, they dropped me off at the wrong school, and they dropped me off at Clay High School instead of Clay Middle School.
And I just [00:08:00] remember that experience up to this day, even though it’s been like 30 something years, they dropped me off and left. And I’m like standing there and I was supposed to have a friend of mine and the ESL teacher meets me up front and I’m standing there in front of that high school and everybody goes in and I’m still there wondering what’s going on.
Like, where are they? Cause I didn’t know that I was at the wrong place. I’m like, I started to figure out that’s not where I’m supposed to be. And then I was like, Oh, what am I going to do? I don’t speak any English. I, there’s no cell phones at the time. So I think, I can’t remember exactly what happened, but I think I ended up going inside.
They figured that I was lost. And I think there was like a Russian speaking student who was, at that school who was able to help me. And figure out what was going on, that I was at the wrong place. And they let me call my brother in law and I got him to pick me up. Even though it was hard for me to explain to him where I was because I didn’t [00:09:00] know.
He was like, where are you? What neighborhood are you in? I have no idea. So that was my first day of school in the U. S.
Justin Hewett: I can imagine that with you. What an experience. And I was sitting, I was thinking, taking the taxi to the other side of town to go get, go to school. Today, if you used an Uber, you would put you, you’d be able to make sure that you were going to the right spot, right?
Tatyana Vdovina: Yeah. Yeah. But yeah, I’m done. You’d have a cell phone and probably more than one middle school would have a cell gram. Back in the day in South Bend, Indiana, I feel like there weren’t that many English learners and the resources were very limited. So then I finally got to the right school on that day and I started my tenure in U.
S. schools. And once again, at that time I was in the ESL program when I was, it was at the end of seventh grade that I moved to the U. S. It was like, I started school in Maine. Then a month later, the school year is [00:10:00] over. And then in eighth grade, I was in ESL. And then by the time I went to high school, I was somehow in all kind of regular, you Classes without any language support, even though I’ve been in the U.
S. for just a little over a year at that point. And I think part of it is, I think at that point there was no ELP, English Language Proficiency assessment out in the field back then. And I don’t remember taking a proficiency test to be able to determine my English proficiency, the academic language proficiency or anything like that.
So somehow it was deemed that I was ready to be out there in all of the regular classes without language support. And I will, I’m, I was.
Mandi Morris: If you were to fast forward from first day of school, dropped off at the wrong place, and what that experience was like for you, how that impacted, I’m sure the start of what school was like, moving through high school and then thinking [00:11:00] about life after high school, how did that feel for you as an English learner?
And coming over in middle school is a really different experience, almost going into eighth grade. Versus learning English when you’re younger or coming in when you were maybe in early elementary school. What was that next chapter like for you?
Tatyana Vdovina: I feel like when you were a kid. You just take things as they come, like you don’t overanalyze, I think, which is probably a blessing.
But yeah, like thinking back on my time in middle school and high school, it was like a tough time socially and academically. I remember having a tough time with understanding academic language and doing my homework at home and just having to sit with a dictionary and having to translate every sentence.
other words sometimes because I didn’t know, even though I must say that I did have a lot of privilege in terms of having had pretty good education in my background coming from my country. I feel like the education there was [00:12:00] pretty, pretty solid. And when I came to the U S I was on grade level or even probably above grade level in some subjects academically.
For me, it was much easier than for. Some students who come in with limited interrupted formal education are slave students who don’t have that. And for them, that transition is so much, that much harder. So for me, it was in terms of academic achievement, it was probably easier. And I was a pretty okay student.
I wasn’t like a straight A student. I was able to keep up with most academic work, even though it was difficult in terms of not knowing the language. I remember I had to transfer out of some. Like maybe it was like an, like a history class because I just couldn’t keep up with all the reading and all of the kind of like also background knowledge that was like hard as well.
Not having the schema when you’re in high school and you’re taking like social studies, government, history, and you have no idea what they’re talking about when they start [00:13:00] talking about this different subjects that all students have heard from probably when they were in elementary school.
Mandi Morris: Wow.
That is such a good point. I wonder now. You’re bringing up the connection between content and language and how your language, your academic vocabulary, your academic language, and your background knowledge wasn’t where it needed to be to feel proficient or success in those history classes. It’s really interesting to think about all the background knowledge.
I’m wondering how does that inform the work that you do now? How do you think about that when in the resources that you develop for English language specialists?
Tatyana Vdovina: Yeah, I think that is something that we always, you know, that we always mention and that we always try to drive home. Unfortunately, I feel like right now the field has a lot more awareness of these different aspects that can be challenges for English learners.
When I, prior to my role here at Manhattan Strategy Group, I used to work at the [00:14:00] Center for Applied Linguistics, which is, I think, how I met you guys, through Maria, who now has the position that I had. Yeah, and so I used to do what she does, do professional development for teachers in different schools around the country, and that is something that we talk to them about, making sure that that background knowledge, that schema, that’s also on top of the language barrier that some students have also in addition then to some students that may have interrupted formal education, the schema that like that knowledge of the world.
that students bring to US schools is very different, right? Even though they may have very rich knowledge of their own background. And I felt like I did. I’ve always been like an avid reader and I’ve read all these books like Alexander Dumas and all of like different books that people in the US don’t normally read.
Kids my age. But when it came to US history, the civil war, like all of that stuff, I was completely [00:15:00] lost and. I feel that is something that, that you really have to keep in mind when you’re teaching and when you’re assessing your students, like you have to give them background information so that they can make that connection.
Otherwise, I feel like a lot of English learners, they can be, especially in subjects like social studies or ELA even, they can be in a vacuum because they can’t make that connection. That connection to what they’re learning in the classroom to what they know. And we know that unless that connection is made, a lot of knowledge is not retained.
Justin Hewett: So I want to go back. I couldn’t agree more as far as there’s needing to think about all the varied backgrounds that students are bringing into the classroom. But I want to go back to Tatyana who immigrates to the United States, has this experience where she’s taken to the wrong school. And now, and then we have that experience that you go through a year and a half later, you’re no longer receiving language services.
What is your experience from there? It looks like you’d stay [00:16:00] close to home to go to university and you went and got your history degree. How do you go from there to being at Incella and creating these amazing resources and tools? I now have a small piece to it. Cause I know you, you ended up at the center for applied linguistics at Cal.
How did you get there? Walk us through that journey. If you don’t mind, I want to hear about it.
Tatyana Vdovina: After I graduated high school, I went to Indiana university and at that time, that was 1998. So by that time I’ve only been in the U S for about five years. Yeah. And I got into the university, but now thinking back, I was just lost.
I decided to major in history and I did, but then I started actually taking a lot of classes in the school of languages there. So I actually got a minor from the Russian and East European Institute. I took a lot of classes in Russian, East European literature, language, like film, and that kind of like always interested [00:17:00] me.
And then when I graduated with my bachelor’s degree, I figured that it probably like I needed to go to graduate school, like I, I would not be able to really find a good job with that degree, just having a bachelor’s. And so I started applying to various programs and I did apply to some like Russian kind of language and literature programs.
And then I, shortly before finishing my undergrad, I visited the Washington DC area and I actually really liked it here because I had a friend that used to live here and I was like, Oh, I really like this area. I want to get out of Indiana, live in a bigger city because that it was always a little challenging for me to live in like a, like small town.
I, I’m more of a bigger, city person. And so I was like, yeah, like, why don’t I look at schools here? And so I was looking at the University of Maryland and they had a second language acquisition program, SLA. And so I applied to this program and I got [00:18:00] in and they offered me a TA ship, teaching Russian, actually, to undergrads.
Yeah, I moved here. I started my master’s degree and it was mostly At that point, I was mostly focusing on like research and Russian language acquisition. So that was what my master’s thesis was on. It was in how students who are learning Russian, how they form this system of learning Russian verbs. And there was like a system that my advisor was working on for me.
teaching the verbs by a special system that should develop and which we use in our classroom. And so we analyze data from that. And then after I graduated with my master’s degree, it was like 2005, I believe. I actually started a PhD program there at the University of Maryland, also in second language acquisition.
I, I was there for four years, I did finish all the coursework, but I did end up leaving without finishing, getting my, writing my dissertation, because it just seemed like that program was not, didn’t turn [00:19:00] out to be the right fit for me.
Mandi Morris: It’s so interesting to hear the connection. I did want to say it’s ironic that history was.
class in high school that you felt like, I didn’t have the background knowledge to make this connection. And then you chose history
Tatyana Vdovina: for a bachelor’s degree. That is true. But see, I think because I’ve always loved history, but, and it was honestly out of nowhere. I was like, oh, I like reading historical fiction.
Like, why don’t I major in history? But I think, yeah, American history was probably the most challenging for me. Government and all of that kind of stuff.
Mandi Morris: You decided to leave, did those four years, decided it wasn’t the right fit for you. And then where did you pivot from there? What was your next move?
Tatyana Vdovina: So while I was in the PHC program, I worked as a TA on this project called the Linguistic World as a Proficiency.
And that was actually an assessment project and like a proficiency assessment in various different languages, one of them being Russian. And we were actually creating an [00:20:00] assessment for students, individuals learning Russian, usually for the getting ready for work in the government, like the Defense Language Institute students and things like that.
But, and I also, at that same time, while I was in graduate school, I took a course that was actually about assessment. And that, I feel like that was like the turning point for me and getting really interested in assessment and language assessment and particularly because Even though I was working on an assessment project when constructing assessment, I didn’t know anything about assessment.
I didn’t know about the concept of validity or reliability or objectives or anything like that, item difficulty, item discrimination, and then I took this course and I was like, Oh, like now there are all these concepts that I should be thinking about. As I’m creating this assessment, but the assessment, we had a statistician who was doing all these item analysis, so it wasn’t a bad assessment, but I feel it was just surprising [00:21:00] to me that as an assessment creator, I had no idea about this concept and after leaving the program, I started teaching English as a second language at the community college here in Maryland, Montgomery County community college, and I taught for About a year, and then they offered, they also have, they still have this, it’s called Sea Soul Ash.
What is it called, like TESOL Education Institute. So it’s a certification program for people who want to teach English either here or abroad. And it’s a non credit program, but it’s a great program. Actually, they have five courses, second, like second language acquisition, teaching reading, teaching grammar, and then assessment.
So they offered me a job teaching the assessment course and just being fresh from that, taking that graduate course, it opened my eyes to the field of assessment. I of course gladly agreed. I taught that course for 15 years, actually, at [00:22:00] Montgomery College. And I feel the longer I taught the course, the more it opened my eyes to the need that is classroom assessment and the gap that educators have in that field, unfortunately, because I feel like a lot of teacher preparation.
program, assessment is not really addressed thoroughly. Fortunately, assessing large scale assessments. We have companies like ETS and Pearson and even the center for applied linguistics would do a great job with large scale assessments, but I feel like classroom assessment is just something that oftentimes.
Is uh, The teachers don’t have the tools to execute that.
Justin Hewett: Yeah. A lot of times they don’t have the background and assessment ends up becoming a dirty word. Where, no, you don’t understand what assessment is if you think it’s a bad thing, but everybody has this connection of summative assessments and how high stakes it is, and it ruins it for all the other assessments.
Come on, summative assessment. You got to, you’re acting [00:23:00] together. No, but seriously, Tatyana, talk to us a little bit about that. Maybe like Why has assessment gotten a little bit of a black eye in education and in ed tech, like in the classroom, right? There are so many different forms of assessment. There’s not just summative assessment people.
There is formative assessment, interim assessment, benchmark assessment. You have all these different ways to approach this. And yet, To a lot of folks, assessment is, has a bad connotation to it.
Tatyana Vdovina: Yeah, I’ve been wondering that myself for many years. And I think it’s because oftentimes teachers feel that the results that the students demonstrate on their assessment, or even what they’re being assessed on, doesn’t have a connection to what they’re learning and what they know.
Right. And I think that’s why assessment has that bed wrap.
Mandi Morris: I’ve seen too over the years in the classroom where teachers feel stretched in a lot of directions where they feel like I have this unit that I have to get through and I have these objectives that I [00:24:00] have to meet and then I have to pause everything I’m doing to do these reading assessments for two days and then for another two days.
So they feel pulled in a lot of directions, whereas if teachers see it as a integrated That formative assessment being integrated into the teaching and informing the instruction. It’s a bit of a shift around how we see assessment. And I agree, Justin, like the summative piece has almost given, like those big end of year summative assessments have almost given like a bad mark for all assessments.
I wonder if you could unpack that a little bit more in teen years teaching an assessment class. That’s incredible. You have so much to teach us.
Tatyana Vdovina: So the class that I taught was pretty much like the fundamentals of assessment. And I do agree that I think one part of assessment is that, yeah, the disruption to the instruction that our teachers experienced with a lot of, and we know that there’s a lot of assessments currently that are [00:25:00] mandated that students have to take.
And like you said, me and Ziya, teachers, they have to get through a certain number of units, a certain amount of content, and then, yeah, they have to stop everything and administer those assessments. And sometimes those are more of proficiency assessments rather than content assessments. So even, I think a lot of times they’re disconnected to some extent from the curriculum that is being taught in school or in the classroom.
And I feel like, yeah, like teachers probably in addition to having to stop what they’re doing in their tracks, they also don’t see how that informs their instruction because the results are not necessarily. providing a lot of feedback to what they’re teaching and what the students are necessarily learning, even though hopefully it does to some extent, but I feel like there is no immediate connection with those types of like big, large scale, standardized summative assessments.
Then when I was teaching my class, I was teaching like fundamentals of assessment to the teachers who are going to [00:26:00] be teaching in the classrooms. What I’ve seen with the students that I had in my class, and I had students of all kinds of different backgrounds in terms of their experience with education, assessment, language instruction.
And instruction in general. So I had very experienced teachers. I had people who are getting ready to retire from government jobs and they want to go and they want to teach abroad, or they want to volunteer with a literacy council. And literally nobody liked assessment. All of them, when they came into my class, they told me that they were dreading it because nobody likes assessment.
And what I tried to do in my instruction, I actually, I tried to change their minds. I tried to let them know that assessment is actually something that can be used as a tool for the educator, right? Something that can give you data and that can give you information about your students. But that. It’s not going to happen if you don’t assess properly, right?[00:27:00]
So what I try to do in my teaching, you have to teach people about what assessment is and what is validity and what is reliability and how do you create good items? Not only so that they can do it for their own formative assessments, for quizzes, but also so that they can evaluate the assessments that may be handed to them.
Okay. And so that they can interpret them in a more appropriate and educated way. And I felt towards the end of my course, almost everybody changed their mind about assessment. We always started with just talking about assessment purposes, because a lot of people, they, they don’t know. Don’t know the difference.
I didn’t know the difference up until I was like almost 30 years old. What is a proficiency assessment? What is a content assessment? What is an achievement assessment? What is an aptitude assessment, right? Open times. What is a placement assessment? Open times, they have assessments, the same [00:28:00] assessment that is used for more than one purpose.
How appropriate is it to use? One assessment for more than one purpose when I worked at Montgomery College as an ESL teacher. I think that kind of further frustrated me with the field of classroom assessment or with the state of classroom assessment because I remember we had this assessment, it was called CASAS, and it was used for more than one purpose, and I always argued that Those purposes were not always appropriate.
So when students started their session, they took this assessment. It was a standardized multiple choice. I think it was reading and writing or reading, listening, and writing, not all language skills. And then we as teachers, we got like a report of where our students were. And the first time I was teaching oh, like I got to look at those assessment results carefully, see where the gaps in knowledge may be for this particular [00:29:00] level.
And then I tried to teach based on the result of the results of these assessments, which were very general, but at that point I didn’t know any better. So I tried to address that. Then, based on the results of the, of that assessment, students were placed in a particular course, like beginner, intermediate, advanced, so that, that was the first purpose placement.
Then, at the end of the 16 week session, the students took this same, probably different form, but the same assessment once again. And what I found the most frustrating, it was not at all what I was teaching during those 16 weeks. Like we may have been talking about, it was life and work skills. So maybe like we’re doing like job interview simulations and filling out like job applications and learning the vocabulary and grammar that is associated with that particular topic.
But then on that assessment, it was something completely different, like different vocabulary, different [00:30:00] grammar. points. And so everybody had to take it. And then based on that, on the results of that assessment, it was determined whether the student would move on to the next level, where they would stay in this level.
Sometimes they would even move down to the lower level. So it was an achievement assessment and once again, a placement assessment. And I thought, like, how is this right? What is being assessed on this assessment doesn’t have any connection to what you were. instructing to the students. So how is it that even though students were learning something, the assessment is telling me that they actually regressed in their English proficiency?
So that was something that was really frustrating to me.
Mandi Morris: Tatyana, I would love to hear your perspective on the state language assessments that our students take across the country. Big complaint that educators have is exactly what you were just talking [00:31:00] about. The content on the state language assessment is so generalized because it’s used nationally, not necessarily replicate match near anything that they’ve been working on in class.
And there has been more and more of a push for portfolio exits between K and 12 and, or a combination of using summative data, summative assessments with portfolio. To exit students from English language development programs. I would really love to hear your perspective on that from an assessment perspective.
Tatyana Vdovina: That’s a great question. And the experience that I have with kind of large scale assessment for English learners in K 12 is Access for ELLs. After teaching in that ESL program, I got a job at the Center for Applied Linguistics while I was still teaching the assessment course part time at Montgomery College.
And for about four and a half years, I worked on the assessment team at the Center for Applied Linguistics. [00:32:00] I wasn’t creating assessments, but I was creating the training materials for the assessment. So the test administration manual, we’re doing field testing, pilot testing for access for ELL, for they had the WIDA model at that time.
That they were developing and also salsa poder, which was the Spanish language assessment that weda was working on. And as in terms of the quality of that assessment, the validity and the reliability side of access for ELLs, I think that it’s a pretty solid assessment. I was actually very.
Pleasantly surprised with the rigor of the different processes that of assessment development that we went through. So there was very careful item specifications that test developers were sticking to. There were very good, processes like the content and bias reviews, field assessment, various statistical analysis were, the item themselves were very good quality.
But [00:33:00] yeah, I think the purposes for which you use that assessment, I think you’re right that it can only be like one, one piece of the, one piece of the puzzle. I completely agree that a standardized assessment, even one of high quality, such as access for ELLs should also should only be. Part of the decision on which language services support services are determined for English learners, because I think things like portfolio or teacher feedback or student interviews, they can really give you a more complete picture of that.
Of the students readiness to be able to be exited from English language services, or maybe just for those students who still need to continue to receive support also taking into account students who may have had interrupted or limited formal education in their backgrounds, I think you can have the best.
assessment of academic language [00:34:00] proficiency, but for students with worse lives that still may not be able to get to the knowledge that they have.
Justin Hewett: Oh, I think so. I think so. It’s good to get your perspective and think about that. And I’m sure that at some point in the future, we will look back and think, Oh my gosh, the only way that you could exit being an English learner was to pass a standardized assessment.
And we’ll think about that and go, wow, how far have we come? What changes have been in store? I imagine some of that’s in the future, but I want to go back to something that you were talking a little bit about before Tatyana, as far as assessment and just that disconnect that I feel like a lot of teachers feel when it comes to assessment, but really when assessment is done the right way, it’s incorporated into curriculum design, built in you’re designing backwards from the outcomes that you work towards.
And it’s just interesting to me. Yeah. Obviously with when building Flashlight [00:35:00] 360, here at Flashlight Learning, we looked at assessment a lot, especially formative and benchmark assessment. And it’s interesting to me how many educators we’ve spoken with who have worked with multilingual students for years, in many cases, decades, who will say something to the effect of, Justin, nobody formatively assess speaking and writing.
I have my ESL endorsement, or I have my certificate and I like nobody ever went into that and taught us about that. And so I’d love your perspective on why is it so hard or why is it ended up being this ambiguous thing to form a formatively assess productive language?
Tatyana Vdovina: I don’t know, I wish I had the answer.
And I think that’s not only the issue with productive language, right? I think it’s the issue with all language assessment. I think it’s because maybe the focus has been so much on the large scale standardized [00:36:00] assessment, which do go through rigorous development. and evaluation to be of good quality. But I feel like, yeah, that focus on classroom assessment for some reason, and I don’t know what that reason is, but it’s not there.
Justin Hewett: And so if you don’t end up formatively assessing language, speaking, writing, whatever it might be, how can you know what that student needs next? If we’re relying on a standardized assessment that we take in January, February, but don’t get the results back till May and then kids go on summer break till August.
They’re back in the classroom, ready to go in September. That is just such a delay. Formative assessment is the good guy in this situation where it informs our next move.
Tatyana Vdovina: Yeah, exactly. It informs the next steps. Yeah, I completely agree with you. And I think, I just think it’s been, unfortunately, and I don’t know why, like I said before, it’s been overlooked.
The topic of assessment in teacher [00:37:00] preparation programs, from what I’ve heard, and I don’t have a teaching license, so I did not go through a teacher preparation program. From what I’ve heard from the field, and I used to work with teachers for almost five years all around the country and actually abroad as well, is that topic is not really addressed.
In their preparation programs and I think for formative assessments, for summative assessments, for anything that you want to do, even asking questions, especially for assessment of the productive skills, you really need to know how to do that. When I was teaching in my Montgomery College programs, even some of the basic things, like when you assess speaking and writing, you have to ask WH questions, information questions.
That was not obvious to many people because They don’t think about it. You don’t know what you don’t know, right? I don’t mean to sound like, Oh, they don’t know anything because teachers know so much, but, and it’s just something that for some reason the system has [00:38:00] overlooked for such a long time.
Justin Hewett: I just, I really appreciate you saying that.
It’s from someone who’s been teaching college courses about assessment, who’s been in this field and in this space and interviewing teachers across the country. It’s just interesting to get your perspective on that because I think at the end of the day. A big piece of it is the fact that we just haven’t had tools to do that work.
And frankly, the reality is if you didn’t have the tools to go build a house, it would be a really hard to go build that house. If you’re just trying to use your hands and go through that and use maybe what you had learned from something else, like tools make a tremendous difference. And they ultimately, they are the difference.
All technology is a tool. And I think sometimes technology gets a little too much credit and sometimes it gets a little bit of a bad rap because sometimes we think, Oh, technology is the solution. It’s the whole answer and it’s not the whole answer. I think it’s a piece if it’s, if it can be a tool.
That we can put in the hands of a teacher that gets a teacher and a student closer together. [00:39:00]
Tatyana Vdovina: Mm hmm.
Justin Hewett: Allows us to assess a student, understand where they’re at today, gather student artifacts. But ultimately, the goal is to get teachers to better understand their students and students to better understand themselves.
And if we can do that, now we’re going to make progress. And so anyways, I do think that’s just interesting that I think a lot of it just comes down to the fact that we haven’t had the tools to do that work.
Tatyana Vdovina: Yeah, absolutely. And like with the tools, when I was working at Cal as a professional development specialist, after it was, I had two positions there, I worked on the Access for ELLs project, then I.
transition to the professional development team, and we would go and work with teachers all over the country. We would train them in SIOP, the Shelter Instruction Observation Protocol, and that has an assessment component. In my work as a professional development specialist, I really try to expand that assessment component because that was like the last component in the training, the [00:40:00] very last.
And it was the shortest, and nobody looked forward to that at the end of a four day training. I tried to do my best to really talk about rubric creation Justin, you just mentioned a tool that can be really useful to teachers in the classroom. Rubrics! You can create your own rubric for formative assessment, right?
One challenge I think that educators may run into with assessing multilingual learners is separating content and language. I think that’s maybe the biggest challenge in classroom assessment for English learners. We think we’re assessing content, but what we’re actually assessing is language. We think we’re assessing language, but what we’re actually assessing is content.
And so I tried to work really hard to educate the teachers about, but you have to separate those two concepts. Otherwise, you’re not going to have a valid assessment of either. And it’s very easy to do, especially with speaking. You can create a content rubric, you [00:41:00] can create a language rubric, or you can create a rubric that addresses both.
And once teachers have these tools, have these ideas, it just makes their work so much easier and that it makes the feedback that comes back to them from their students so much more valuable because they actually validly are assessing one construct or the other. Right.
Mandi Morris: I think it’s something that. is unique.
You get like a T, like a teacher’s edition of your textbook, and it has the summative. Like you’ve got your end of unit assessment that you need to do. And in the mid 2000s, I was working at a school where we got the focus of the school was teachers understanding assessment, both summative and formative.
They developed protocols where in the grade book you had to have this many formatives before you could have a summative. So it was really driven even by numbers A student couldn’t fail a summative if you didn’t have formatives leading up to it. Showing the student’s [00:42:00] progress throughout the formatives.
And how did the instruction impact to try to help the students have success on the summative? So it was it was a little bit top down system and, but they were really trying to. ingrain in us that formative has to be a part of your daily practice in the classroom. And I walked away from those trains before the school year.
I’m like, okay, I understand what formative, I’ve got to have them throughout this, my, my day, my week. I understand summative, right? But what does formative look like? What? So I have a terminology in my head. But then what is the practice in my classroom? Is formative kneeling down next to students, listening to them talk and noting where there are gaps and where there are gains?
Is formative a quiz? Could you talk a little bit about when you are teaching formative to people, we understand the theory behind it, but in practice, what does it look like in the classroom? And if you could describe. Best practice for formative assessment. How would you describe that? [00:43:00]
Tatyana Vdovina: Any assessment can be a formative assessment or a summative assessment, right?
So the assessment itself, it, that doesn’t really matter, right? Because if we have a quiz that we’ve been teaching for three days about a topic, and then we have a quiz that we give to the students, let’s say it’s multiple choice. We give it to them. We grade it and then we forget about that. You’re right.
And maybe we take the grade and that makes, contributes like 5 percent of the final quarter grade or marking period grade. That’s a summative assessment, right? What you do with the results that can also be a formative assessment, right? So if you have a quiz, you have for your. English learners, maybe you do item modifications, right?
Maybe you provide a word bank for your English learners, and then you look at the results and, oh, like I see that on this item, on this particular item, like 90 percent of my students, they answered it correctly on that item. Maybe 65 didn’t get it. You look at your items and see, oh, is [00:44:00] something wrong with this particular item?
Maybe the district are confusing and my students were confused. And then you take the results and then you reteach the topics that your students didn’t understand. go over the results with the class, and you incorporate it in your teaching and students learning. That’s what makes an assessment a formative assessment.
So it can be the exact same assessment, but what you do with the result of the assessment will determine whether it’s formative or summative. Because I think formative, it’s the idea of formative, that it’s still like in the process it helps the students form knowledge. Contribute to their acquisition of content or their acquisition of language, right?
With English learners, I think it’s especially important to formatively assess any chance that you get in the classroom, right? To really, and I feel like that begins with assessment construction. If teachers are creating their quizzes themselves, keeping in mind the language [00:45:00] difficulty, right? If you’re assessing content, if you’re a content teacher, you’re assessing content, but you have English learners in your class.
What can you do to alleviate the language difficulty, right? Because if the students answer a question, it doesn’t matter what the format is. It could be an essay question. It could be a verbal question. It could be a multiple choice question, true, false, whatever. If somebody answers. an item incorrectly because they didn’t understand the language, but they have the content knowledge, then you as a teacher are not getting the data that you need.
You’re assessing, you’re forming the wrong, you’re making the wrong evaluation of that student. of that student. I feel like that idea of having a formative or a summative assessment that begins with assessment construction. On the other hand, if you as a teacher, even if you’re creating a quiz or if you have maybe a quiz that you already have in your teacher handbook, see what kind of modifications can you make for your English learners, right?
So maybe simplify [00:46:00] the language if the objective is to assess the content of certain content point. Simplify the language so that the students can understand the content. Provide a word bank, provide, if you have the tools, if you have the ability to provide translations of key terms, especially those tier two verbs to the students, because those are the hardest.
And that way you can really get as a teacher feedback and data about your students knowledge and then address them in further instruction or your feedback. To your students or other stakeholders, I think that’s what really would contribute to best practices and affirmative classroom assessment is really thinking about what you’re assessing, the objectives that you’re assessing, language or content, and then creating tools like rubrics, depending on, of course, on the skill that you’re assessing and providing assessment modifications if needed for English learners.
Mandi Morris: So, I’m a 6th grade teacher, and I give my [00:47:00] summative on ancient history for Egypt, ancient history for Egypt in 6th grade, and half my class fails. What do I do next? What does that tell me and then what are my next steps that I need to do with that assessment?
Tatyana Vdovina: So it depends on the format of your assessment, right?
If it’s all multiple choice, I would suggest maybe redoing it and adding some varied item formats. So not only doing multiple choice, but also maybe doing an essay question or an oral Q& A with the students or some sort of a quiz. project based assessment as well. So considering all these options, and just to diverge a little bit on the assessment format issue, with English learners coming from all types of different educational background, they may not be familiar with the assessment format itself.
When I was in school as an English learner, when I first time I was handed a scan, I don’t know if you remember the Scantron, [00:48:00] I had no idea what to do with it. No idea. So, just being cognizant of, are your students familiar with your, with the format in which you’re assessing them? Then, if most of your students, you said 90 percent of your students failed, it could be two things, right?
It could be an artifact of your assessment, or it could be that they didn’t get the material. Of course, that, that’s always a possibility. Let’s see. Starting with examining the quality of your assessment, and that’s the assessment that you as a teacher created. Created, that’s the first thing. If I see some extreme result, most students failed, and you as a teacher would know because you’ve been teaching that content.
So if you see that stuff, kids are getting it, you would, and then everybody fails on the assessment. That’s probably something is going on with assessment. So that’s what I would do. I would go back and I would examine the assessment. I would, first of all are the items that you [00:49:00] created, do they address the objectives that, that you want to get information on?
Because especially with constructed, sorry, selected response items like multiple choice, true, binary choice. One item can only address one objective, right? And that’s OpenChimes. That’s the downfall that when we create multiple choice items without Knowing the fundamentals of item construction, you can, there’s so many things that can go wrong and you can just not get good data.
Also then that idea of am I assessing content? Am I assessing language? If you have a lot of English learners of various proficiency levels in your classroom who have failed this exam, but you knew that during instruction, they seemed like they were understanding things, I would. Really look at the language difficulty of your assessment, and like I mentioned before, see how you can simplify your items.
If it’s a constructed response, maybe like an essay [00:50:00] question, what can I do? Once again, word back, word bank, sentence starters, or paragraphs, visual support for students, or something like that. Something where they can express their knowledge. And not, but where the language is not standing in their way.
And like I mentioned before, it’s all like a proper rubric. Do you have a proper rubric? Is your rubric addressing content or is your rubric addressing language? Are you grading grammar and punctuation on a content question? If you are, then it’s not a valid assessment.
Justin Hewett: That’s right. That’s right. Oh, that’s it’s so great to hear you unpack that and work through your thinking of how you would approach it in that situation.
Frankly, I just love talking through assessment and getting your perspective and working through it. Right. One of the things I think about when I think about formative assessment is that a big piece of it is just as a teacher, you’re just doing your best to learn about your student. And yes, sometimes how you [00:51:00] build the assessment is really important and sometimes just figuring out a way to assess something is what’s important.
It’s just figuring out a way to get some feedback. Um, does the student understand what I’m trying to teach and can I capture that somehow so that I can use that when I’m sitting down lesson planning three days from now for the next unit? Do I have that data in front of me? And that’s one of the things that we really worked hard with Flashlight to build was to make sure that tool would help capture some student responses that they’d have that rubric that they could come back and refer to it and look at it and hopefully make it easy.
But one of the things I would really love to do with you, I know that we’re getting to the end of our time here, Tatyana, you are building tools for teachers in CELA right now that we would love to learn a little bit more about. We’d love for you to maybe walk us through what some of those tools might look like that They have access to, how to access them.
What does that look like? And really what, what the goal and directives are for your team as you’re building these tools.
Tatyana Vdovina: [00:52:00] Yeah, absolutely. So Ancela has just a variety, a wealth of resources for educators as English learners. It’s not really assessment oriented. So that’s something that I really miss.
And that’s why I really wanted to talk to you guys about that because I no longer am teaching the course. And that is just something that I’m really missing now is being able to talk about assessment and to give teachers those tools and fundamentals of assessment.
Justin Hewett: Maybe what we need to do is set up a Zoom class and teach us all about.
And instead of it being through the college there, it’ll just be a master class.
Tatyana Vdovina: Yeah. Like a workshop. Yeah. I’d love to do something like that. Absolutely. But anyway, regardless of that, Ancela is an awesome collection of resources for educators. The mission of Ouela is to actually promote multilingual education and to bridge the gaps, the achievement gaps for English learners, for [00:53:00] multilingual learners, to really enhance family engagement.
And to really promote multilingualism for all students. And so that’s what the resources are all about. I feel like my favorite Ancela resources are the fact sheets. So those are, they can be found at the Ancela website. And these are two page publications on various topics. That have to do with education of English and multilingual learners.
So the one that I’m actually currently working on with our partners is English learners and career and technical education. So we’re about, hopefully that’s going to be published in a couple of months. And that just gives you a lot of the facts, they give you just data, right? So it’s really very high quality.
federal resource data that just tells you about the state, for example, of English learners in CTE. We have the percentages of concentrators, the percentages of English learners who participate in CTE. What are the gaps [00:54:00] between English and non English learners who are CTE participants and concentrators?
And so these are really great, really quick resources with a lot of visual support where educators get look and see, Oh, like what’s going on? I’m a CTE teacher. Like I have English learners in my class. I can just look at this resource and kind of gather like the state of things for English learners in CTE.
We have fact sheets about English learners who study. speak various home languages, like English learners who speak Arabic as a home language, Mandarin as a home language, who speak Spanish as a home language. And that just gives you data about who these students are, which states they’re concentrated in, how are their populations increasing, decreasing, things like that.
These are pretty good examples. Honestly, my favorite resources, those two page fact sheets, they’re so great for professional development purposes, for just education resources, for if somebody’s looking, they want to see, oh, where’s my state? For English learners who are Arabic speakers, you can find data on that.
We [00:55:00] do a lot of webinars, like I said, on various topics. We’re working currently, we’re going to have a webinar at the end of May about the community language schools and how those institutions contribute to promoting multilingualism in communities. So that’s going to be a really interesting webinar. A lot of resources.
such as webinars, data visualizations, fact sheets, infographics, some dual language and bilingual education, because those are the programs that are, that really had been working well for English learners, multilingual learners, because they can provide instruction in their home language, they can bridge the achievement gap, the opportunity gap.
So we’ve been doing a lot on how administrators can support dual language programs, what are some teacher preparation pathways. We’re going to have a five part series in the future. somewhere that’s going to address a lot of different aspects of dual language education. So I love that topic as well.
Justin Hewett: Oh my gosh.
I’m on the website [00:56:00] looking at all these fact sheets. There is a, yeah, there is a lot of really good information here. For example, raise the bar, create pathways for global engagement in English. What is high school graduation rates for English learners around the country? The top languages spoken by English learners in the U.
S., demographic trends, English learners and homelessness. Anyways, this is really interesting as I look at this there’s a lot of really good data on each one of these fact sheets.
Tatyana Vdovina: Yeah, and I feel like the products that Ansela generates, they, in addition to being great resources for educators of English learners, for families of English learners, I think they really address the equity for English learners and bringing the different challenges, making English learners from various backgrounds visible to educators and to the community in large.
I think they really do a great job with that, with promoting equity for English learners and multilingual learners. [00:57:00]
Justin Hewett: There is so much goodness on here. I’m looking at the, I guess this is maybe an article of some sort in the resource library. There is, yeah,
Tatyana Vdovina: the resource library. It has tens of thousands of resources that are all, they’re all open access.
Everything is free. You don’t have to subscribe to anything. This is just something that is just a repository. Wonderful resources for English language educators. students and families. There are toolkits. We have three toolkits that do provide really great guidance. There’s the family toolkit, the newcomer toolkit, the English learner toolkit.
The family toolkit and the newcomer toolkit are the newer resources that feature the latest kind of guidance, the latest data, the latest recommendations, snapshots of boarding English learner families for English learners who are newcomer students. I love it. It has. A lot of really great stuff in there.
Mandi Morris: Yes, and that Newcomer Toolkit, if anyone has newcomers and you [00:58:00] have not used that resource yet, put it at the top of your list this summer when you have time. Check it out. Um, that is a really wonderful resource and I hear out of the Family Toolkit, but especially that Newcomer Toolkit, I’ve heard professionals talk about a lot over the last few years.
Tatyana Vdovina: Wonderful. Yeah. It’s just has been updated in the last, I think in the last two years, we updated it.
Justin Hewett: Tatyana, what a pleasure to spend all this time with you. Thank you so much for talking with us about assessment and about the great work you’re doing in CELA. It’s so great to hear your experience in coming to the U.
S. and how formative that was for you in choosing the career that you have and doing all the great work that you’ve done. We really appreciate all the work that you’re doing to serve our multilingual learner community, the educators and the students in this space and you’re making a big difference.
What is one maybe piece of advice or that you might share with somebody that’s just beginning in their journey of serving multilingual learners, or maybe it’s something that you wish you had [00:59:00] known back then or whatever it might be like, what’s something you might share with that young educator, if you don’t mind.
Tatyana Vdovina: I would say get to know your students because the student will always forever remember the worst and the best teacher that they have had, especially the best, because I still remember my teacher from middle school and actually high school, Mr. Belkin. I don’t know if he’s listening, but shout out to him.
He was an ESL teacher in the middle school that I attended when I just came to the U. S. And then when I went to high school, he actually Worked at that high school as a Spanish teacher and I took his class and he was somebody who, he really got to know his students. He asked questions, he was genuinely interested and that is just something that students will remember for the rest of their lives.
They will never forget kindness, they will never forget, They will never forget genuine interest.
Justin Hewett: That is the cherry on top. [01:00:00] You might not like giving advice very much, but that is some of the best advice I think I’ve ever heard, because I think it’s about understanding your students, understanding their needs, understanding where they’re coming from.
It’s been said that the only thing that matters is how much you care about them. And how you made them feel. And I think that Mr. Belke obviously did a really good job of helping you feel like you mattered. You were important. You were seen. You were heard. And you are going to get better at this. And that really resonates with me.
Tatyana, thank you so much for being here. Being our guest on this ML Chat podcast. So fun for us.
