1 Student Learning Survey Constructs

The Student Learning Survey has been conducted every year in B.C. public schools since 2001 for Grades 4, 7 10, and 12 students. It is the only source of province-wide information on students’ learning experiences. The data from the survey is used by the Ministry of Education and Child Care, Ministry of Health, districts and schools, and the Representative of Children and Youth, to help improve the well-being of youth in B.C., improve student outcomes, and support school planning and programs.

In 2023, the Student Learning Survey (SLS) is composed of fifty-seven (57) questions for Grade 4, seventy-two (72) questions for Grade 7, and eighty-one (81) questions for Grade 10 and 12. To further aid data use and analysis, SLS constructs were developed using the 2021/22 and 2022/23 SLS data. Constructs are summary measures composed of survey questions and measuring unobservable concepts or attributes. The purpose of these constructs is to make interpretation easier and improve research reliability by minimizing error. The constructs also illustrate where the SLS could be developed as they summarize exactly what the SLS does and does not measure. Several statistical methods were used to create the constructs, increasing their reliability and validity, reducing measurement error, and removing subjectivity in how the questions are grouped together (Heckman et al., 2013; Thompson, 2004). The constructs are further categorized into five summary constructs which are aligned with the overarching areas that the student learning survey was designed to measure.

This document aims to provide users of the SLS data, reports, and dashboard an overview and general idea of the constructs, how each construct is measured and the possible uses that they provide. The succeeding sections are outlined as follows:

2 Methodology

In 2021, the SLS underwent major revisions to better align with and support BC’s vision of developing an “education citizen.” Revisions also sought to ensure the tool continues to provide robust assessments on students’ progress and available support in the areas of intellectual, human, social, and career development. This includes aspects in the development of BC’s core competencies, essential learning, literacy, and numeracy. The development of the SLS constructs included the following key approaches:

For the purposes of this guide, the succeeding sections will be focusing on the factor analysis used to create the constructs, the composition of each construct, and the calculation of construct scores.

Continuous evaluation, review and refinement of the constructs are currently ongoing based on the feedback we receive from partners, districts, and stakeholders.

2.1 Factor Analysis

A statistical technique called factor analysis was used to choose constructs based on statistical relationships, reducing the subjectivity of our approach. Exploratory factor analysis uses statistical relationships to group questions together, explaining themes in the data without any predefined relationships. After exploring the data and grouping questions together based on the results and theoretical framework, confirmatory factor analysis was used to validate the reliability of the construct. Confirmatory factor analysis tests how well the constructs fit relationships in the actual way respondents answered questions.

The constructs summarize all of student experience, so all information relevant to the theoretical framework was used.1 Following literature recommendations, model fit criteria and a theoretical framework was used to identify which questions are related and how related they are (Borghans et al., 2008; DiStefano et al., 2019; DiStefano & Hess, 2005; DiStefano & Morgan, 2014; Enders & Bandalos, 2001; Hu & Bentler, 1999; Li, 2016; Savalei, 2010; Shi et al., 2020; Shi & Maydeu-Olivares, 2020; Thompson, 2004; Voorhees et al., 2016; Widhiarso & Ravand, 2014; Xia & Yang, 2019).2

Since constructs were created to summarize how students’ attributes and experiences are related, one-dimension constructs are developed. As a result, causal relationships are discarded; for example, sleep and mental health are highly related, but not because they are the same – rather, sleep affects mental health and mental health affects sleep. Importantly, because all relevant questions are included and one-dimensional constructs are identified, one and two-question constructs exist. One and two-question constructs are interpreted as identifying unique but smaller attributes of a student’s experience. The user is cautioned that one and two-question constructs are typically less reliable.

The construct names were selected based on the information from the survey questions that compose them. For example, responses to “I would like to go to a different school”, “Do you feel welcome at school?”, “Do you like school?”, and “Is school a place where you feel you belong?” are combined to identify the “Belonging” construct. If questions belong to the same construct, it indicates they measure the same underlying attribute or concept.

Table 1 presents each construct (in italics). The questions listed below each construct are captured in the construct if there is a check mark, or, if there are 3 or more questions, a factor loading.3 Factor loadings provide an indication of the strength and direction of the relationship between the survey question and the underlying attribute the construct represents. The higher the factor loading value, the stronger the relationship between the construct and the survey question (i.e., an increase in the underlying attribute is associated with an increase in the survey question response).

Considerations:

  • Consistent construct definitions were used across all grades to improve interpretability.
  • Some questions, such as student safety, are anonymous. Since anonymous respondents’ school is the only information that can be identified, school-level constructs use school average question responses and the same factor analysis process as other constructs.4 These constructs are denoted by “School Level” in Tables 1 and 2.
  • Some questions are unrelated to any other question. These questions are interpreted as identifying unique constructs, as shown in Table 1.

Table 1: Construct Composition and Factor Loadings (Printable)

Questions Grade 4 Grade 7 Grade 10 Grade 12
Curriculum
Assessment
S52- Do you know what things are considered when your work is marked? (Gr 7/10/12) ✓ ✓ ✓
S54- As part of marking your work, are you shown examples of what excellent, good, fair, and poor work look like? (Gr 7/10/12) ✓ ✓ ✓
Cognitive Skills
S16- Are you learning ways to think of new ideas? (Gr 4); At school, are you learning creative ways to think of new ideas? (Gr 7/10/12) ✓ ✓ 0.72 0.69
S17- At school, are you learning to be a critical thinker? (Gr 7/10/12) ✓ 0.72 0.74
S21- I am satisfied that in school I am learning basic reasoning skills that I need for the future. (Gr 10/12) 0.68 0.71
RMSEA 0.00 0.00
Cronbach’s Alpha 0.775 0.78
McDonald’s Omega 0.775 0.78
Environmental Education
S33- At school, are you learning about how people change the world around us? (Gr 4/7); At school, are you learning about how human activity affects our environment? (Gr 10/12) ✓ ✓ ✓ ✓
S60- At school, are you learning how to do things to care for the environment? (Gr 7); At school, are you learning how to do things to care for the environment (recycling, waste reduction, water conservation)? (Gr 10/12) ✓ ✓ ✓
Extracurricular Load
S126A- I feel like I have too many things to do outside of school. (Gr 4) ✓
Extracurriculars
NQ5- Do you go to any clubs, dance, sports, or music classes outside of school time? (Gr 4/7); At school, do you participate in activities outside of class hours (for example, clubs, dance, sports teams, music)? (Gr 10/12) ✓ ✓ ✓ ✓
S62- Do you have opportunities to learn in places outside of school? (Gr 7/10/12) ✓ ✓ ✓
Feedback
S40- At school, are you provided with feedback on how you could improve your learning? (Gr 7/10/12) ✓ ✓ ✓
Indigenous Participation
S133- Do you participate in any ongoing Indigenous (First Nations, Inuit, Metis) activities outside your school day? (Gr 4); At school, do you participate in any ongoing Indigenous (First Nations, Inuit, Metis) programs or activities? (Gr 7/10/12) ✓ ✓ ✓ ✓
S73- Do you participate in any Indigenous (First Nations, Inuit, Metis) activities? (Gr 4); At school, do you participate in any Indigenous (First Nations, Inuit, Metis) celebrations or activities? (Gr 7/10/12) ✓ ✓ ✓ ✓
Indigenous Study
S131- At school, are you learning about local First Nations’ language(s)? (Gr 4); At school, are you being taught the local First Nations’ language(s)? (Gr 7/10/12) 0.5 0.52 0.53 0.57
NQ20- At school, are you learning about Indigenous Peoples (First Nations, Inuit, Metis)? (Gr 4); At school, are you being taught about Indigenous Peoples (First Nations, Inuit, Metis) in Canada? (Gr 7/10/12) 0.61 0.77 0.76 0.82
S132- At school, are you learning about local First Nations? (Gr 4); At school, are you being taught about local First Nations? (Gr 7/10/12) 0.78 0.89 1.01 1.04
RMSEA 0.00 0.00 0.00 0.00
Cronbach’s Alpha 0.693 0.767 0.756 0.789
McDonald’s Omega 0.705 0.785 0.782 0.812
Learning
S48- At school, do you get to work on things you are interested in? (Gr 4); At school, do you get to work on things you are interested in as part of your course work? (Gr 7/10/12) 0.67 0.71 0.62 0.63
C4- Do you have the opportunity to apply the concepts you learn in school to other areas of your life? (Gr 10/12) 0.52 0.51
S49- Are you taught to improve how you learn? (Gr 4/7); At school, are you taught to take ownership or control of your learning (for example, marking your own work, making decisions about what you want to study)? (Gr 10/12) 0.42 0.58 0.6 0.61
S98- Do you feel you have choice about what you are learning? (Gr 4); Do you feel you have a choice about what and how you learn? (Gr 7); At school, I provide input into what I learn, and how I learn (for example, suggesting topics that interest me, and choosing my projects) . (Gr 10/12) 0.66 0.73 0.5 0.53
S58- At school, do you get to show your work in creative ways (pictures, writing)? (Gr 4); Do you have chances to show what you learned in different ways (pictures, models, writing)? (Gr 7); Are you taught to show your learning in different ways? (for example, pictures, models, written work) (Gr 10/12) 0.43 0.47 0.49 0.5
RMSEA 0.054 0.048 0.032 0.029
Cronbach’s Alpha 0.619 0.717 0.712 0.729
McDonald’s Omega 0.627 0.724 0.714 0.73
Learning Skills
S114- I try my best in learning the lessons taught at my school. (Gr 4) ✓
S56A- Do you set a goal when learning something new? (Gr 4) ✓
Learning self care
S7- Are you learning how to care for your mental health? (Gr 4); At school, I am learning how to care for my mental health. (Gr 7); At school, I am learning how to care for my mental health. (for example, anxiety or stress management, anger management, relationship skills) (Gr 10/12) ✓ ✓ ✓ ✓
S8- Are you learning how to care for your body? (Gr 4); At school, I am learning how to care for my physical health. (Gr 7); At school, I am learning how to care for my physical health. (for example, getting healthy food, exercise and sleep) (Gr 10/12) ✓ ✓ ✓ ✓
Lessons
S56- At school, do lessons begin with a review of the last lesson? (Gr 4); At school, do lessons often begin with a review of what was done in the last lesson? (Gr 7/10/12) ✓ ✓ ✓ ✓
Literacy
S20- I feel I am getting better at reading. (Gr 4); I continue to get better at reading. (Gr 7); I continue to get better at reading (understand main ideas, interpret information from text and other sources). (Gr 10/12) ✓ ✓ ✓ ✓
S22- I feel I am getting better at writing. (Gr 4); I continue to get better at writing. (Gr 7); I continue to get better at writing (use correct grammar, organize and compose ideas, revise, and edit). (Gr 10/12) ✓ ✓ ✓ ✓
Media
S35- Is your school helping you to access and analyze media content? (Gr 7); Is your school helping you to access, analyze, and evaluate media content? (Gr 10/12) ✓ ✓ ✓
Numeracy
S19- I feel I am getting better at math. (Gr 4); I continue to get better at math. (Gr 7); I continue to get better at math (apply formulas, interpret data, solve problems). (Gr 10/12) ✓ ✓ ✓ ✓
Future Preparation
Idea of Future
C3- I have an idea of what I want to do when I grow up. (Gr 4/7) ✓ ✓
Life Preparedness
Q42- Are you satisfied that school is preparing you for a job in the future? (Gr 10/12) 1.02 1.01
Q43- Are you satisfied that school is preparing you for post-secondary education? (Gr 10/12) 0.86 0.83
S6- I am satisfied that in school I am learning basic life skills that I need for the future (finances, setting goals, independent living) (Gr 10/12) 0.82 0.83
RMSEA 0.00 0.00
Cronbach’s Alpha 0.833 0.817
McDonald’s Omega 0.837 0.821
Interpersonal Skills
Collaboration
S39- Do you get to work together on projects with your classmates? (Gr 4); At school, do you get to work together on projects with your classmates? (Gr 7/10/12) ✓ ✓ ✓ ✓
S51- Do you work well with others on group projects? (Gr 4); Do you collaborate well with others on group projects to achieve goals? (Gr 7/10/12) ✓ ✓ ✓ ✓
Diversity
Q18- At school, do you respect people who are different from you (for example, think, act, or look different)? (Gr 7/10/12) ✓ ✓ ✓
S29- At school, I am learning to understand and support human rights and diversity. (Gr 7/10/12) ✓ ✓ ✓
Emotional Development
S10- When you make a choice, do you think about how it might affect others? (Gr 4/7); When you make a decision to do something, do you stop to think about how it might affect other people? (Gr 10/12) ✓ ✓ ✓ ✓
S14- I am satisfied that in school I learn how to express emotion, and deal with emotional problems that I may face in the future. (Gr 10/12) ✓ ✓
Listening
C1- At school, I am learning to listen well. (Gr 4) ✓
Planning
S38- I plan my learning based on my goals. (Gr 10/12) ✓ ✓
Problem solving
S57- Can you explain to others how you solve problems? (Gr 4/7); Can you explain your thinking and describe to others how you solve problems? (Gr 10/12) ✓ ✓ ✓ ✓
Resiliency
C2- When you have trouble doing something, do you keep trying? (Gr 4); When I am facing difficult tasks, I keep trying until I succeed. (Gr 7/10/12) ✓ ✓ ✓ ✓
C5- I believe I can be successful at almost anything I set my mind to. (Gr 7/10/12) ✓ ✓ ✓
Social Skills
S116- I like making new friends and meeting people at school. (Gr 4); I am satisfied with my ability to make new friends and meet people at school. (Gr 7/10/12) ✓ 0.62 0.58 0.6
S25- At school, I am learning to communicate well. (Gr 7/10/12) 0.72 0.8 0.77
S27- Are you learning how to solve problems with others in peaceful ways? (Gr 4/7); At school, I am learning how to resolve conflicts with others in peaceful ways. (Gr 10/12) ✓ 0.52 0.55 0.57
S30- I am satisfied that in school I am learning basic social skills that I need for the future (build relationships, support others). (Gr 10/12) 0.83 0.85
RMSEA 0.00 0.005 0.03
Cronbach’s Alpha 0.658 0.749 0.756
McDonald’s Omega 0.664 0.757 0.761
School Environment
Adult support
Q26A- How many adults do you think care about you at your school? (Gr 4/7); At your school, how many adults do you feel care about you (for example, teachers, counsellors, student helpers)? (Gr 10/12) 0.53 0.75 0.74 0.69
S96- If you have a problem, can you get the help you need from adults at your school? (Gr 4); Are you able to get the information and advice that you need from adults at your school? (Gr 7); At school, are you able to get the information and advice that you need? (for example, from teachers or counsellors) (Gr 10/12) 0.64 0.66 0.64 0.61
S110- Are you able to get extra help with your schoolwork when needed? (Gr 4); At school, are you able to get extra help when needed? (Gr 7/10/12) 0.59 0.63 0.62 0.61
S50- If you do not understand something at school, do you ask for help? (Gr 4); At school, how often are concepts explained in ways that you can understand? (Gr 7/10/12) 0.49 0.5 0.48 0.46
S99- Do the adults at your school welcome your questions? (Gr 4); Do the adults at your school value and welcome your questions? (Gr 7/10/12) 0.54 0.66 0.64 0.61
RMSEA 0.036 0.024 0.012 0.021
Cronbach’s Alpha 0.698 0.773 0.75 0.759
McDonald’s Omega 0.7 0.776 0.753 0.762
Belonging
NQ10- I am happy at my school. (Gr 4); I would like to go to a different school. (Gr 7/10/12) 0.77 0.68 0.61 0.62
NQ16- Do you feel welcome at your school? (Gr 4/7/10/12) 0.67 0.77 0.73 0.71
Q3- Do you like school? (Gr 4/7/10/12) 0.77 0.71 0.67 0.66
S81- Is school a place where you feel like you belong? (Gr 4/7/10/12) 0.79 0.86 0.82 0.81
RMSEA 0.094 0.101 0.091 0.074
Cronbach’s Alpha 0.837 0.801 0.766 0.772
McDonald’s Omega 0.837 0.802 0.766 0.773
Community
S26- I feel that I can make a difference in my community. (Gr 7); I contribute in meaningful ways to my community. (Gr 10/12) ✓ ✓ ✓
Fair Treatment
NQ1- Do adults in the school treat all students fairly? (Gr 4/7/10/12) ✓ ✓ ✓ ✓
Rules
S68- I know how my school expects me to behave. (Gr 4); At school, rules and expectations for behavior are clear to me. (Gr 7); At school, rules and expectations for behaviour are clear (for example, school rules or codes of conduct). (Gr 10/12) ✓ ✓ ✓ ✓
School support
S103- My school provides enough supplies for my learning. (Gr 7); My school provides enough materials and technology that I need for my learning. (Gr 10/12) ✓ ✓ ✓
S111- At school, I have felt well supported when I moved to a higher grade (for example, by having course choices and timetables or calendars explained). (Gr 10/12) ✓ ✓
Teachers
Q8- Who gives you extra help with your schoolwork? (Gr 4); How many teachers help you with your schoolwork when you need it? (Gr 7/10/12) ✓ ✓ ✓
Bullying (School Level)
Q21- Have you ever felt bullied at school? (Gr 4/7/10/12) ✓ ✓ ✓ ✓
Discrimination (School Level)
S82- At school, have you experienced discrimination on the basis of your sexual orientation or gender identity? (Gr 7/10/12) ✓ ✓ ✓
Safety (School Level)
Q22- Do you feel safe at school? (Gr 4/7/10/12) ✓ ✓ ✓ ✓
S108- I feel safe when I am going from home to school, or from school to home. (Gr 7/10/12) ✓ ✓ ✓
Wellbeing
Nutrition
S124- How many times a week do you eat breakfast? (Gr 4/7); How often do you usually eat breakfast? (Gr 10/12) ✓ ✓ ✓ ✓
S125- How many times a week do you eat fresh vegetables? (Gr 4/7); How often do you usually eat fresh vegetables? (for example, in a salad, or fresh-cooked from raw) (Gr 10/12) ✓ ✓ ✓ ✓
Physical Health
NQ12C- On a normal school day, how many hours do you exercise (can include recess, lunch, P.E. class)? (Gr 4/7); On a normal school day, how many hours did you exercise or do physical activities that made you sweat and breathe hard? (Gr 10/12) ✓ ✓ ✓ ✓
S128C- How would you describe your physical health? (Gr 7/10/12) ✓ ✓ ✓
Sleep
S126- How many times a week do you wake up tired? (Gr 4); How often do you usually get a good night’s sleep? (Gr 7/10/12) ✓ ✓ ✓ ✓
Alcohol (School Level)
S117- Do you drink alcohol? (Gr 7/10/12) ✓ ✓ ✓
Food Scarcity (School Level)
C6- This school year, how often have you felt hungry because there was not enough food to eat? (Gr 4/7/10/12) ✓ ✓ ✓ ✓
Mental Health (School Level)
S122- Do you feel good about yourself? (Gr 4/7/10/12) ✓ 0.32 0.34 0.26
S128B- How would you describe your mental health? (Gr 7/10/12) 0.38 0.38 0.42
S120- Does school make you feel stressed or worried? (Gr 4); Does school make you feel stressed or anxious? (Gr 7/10/12) ✓ 0.31 0.33 0.22
RMSEA 0.00 0.00 0.00
Cronbach’s Alpha 0.836 0.823 0.698
McDonald’s Omega 0.839 0.825 0.727
Nicotine (School Level)
S119- Do you use tobacco or nicotine in any form? (for example, smoking, chewing, vaping) (Gr 7/10/12) ✓ ✓ ✓
Note:
Table 1 describes the composition of constructs. Constructs are denoted by italics and summary constructs by bold underlining. The second through fifth columns capture the factor loadings of questions - a statistic measuring the strength and direction of the relationship between the survey question and the underlying attribute the construct represents. If a grade-question cell has a value or check mark, it is included in the construct (not all questions are available in each grade). If a cell is blank, the question is not available for the grade. If a construct has anonymous questions within it, the constructs are measured using school-level average responses (denoted as ‘(School Level)’); in all other cases, constructs use student-level responses. Factor loadings are replaced by check marks for 1-2 questions constructs as they have less meaning. If there is more than three questions, the reader can gauge factor fit using the fit statistics listed below each the set of questions; RMSEA means Root Mean Square Error of Approximation and is a measure of model fit (near 0 is better), whereas Cronbach Alpha and McDonald’s Omega capture construct reliability (near 1 is perfect, above 0.7 is good).

2.2 Summary Constructs

To make interpretation easier, multi-dimensional summary constructs were created. Summary constructs are composed of constructs capturing similar or related topics. They are subjectively defined by construct relationships and how the construct interpretation is related to the framework. Summary constructs simplify the constructs further for ease of use in the dashboard and reporting purposes. For example, the School Environment summary construct is composed of the Adult Support, Belonging, Bullying, Community, Discrimination, Fair Treatment, Rules, Safety, School Support, and Teachers constructs. Readers are recommended to consider them as a starting point to gauge which construct they need to investigate further. In Table 1, summary constructs are identified by bold and underlined headings. In total, all constructs are grouped into five summary constructs. These are Curriculum, Future Preparation, School Environment, Well-Being, and Interpersonal Skills.

Table 2: List of Constructs by Grade

Construct Grade 4 Grade 7 Grade 10 Grade 12
Curriculum
Assessment NA ✓ ✓ ✓
Cognitive Skills ✓ ✓ ✓ ✓
Environmental Education ✓ ✓ ✓ ✓
Extracurricular Load ✓ NA NA NA
Extracurriculars ✓ ✓ ✓ ✓
Feedback NA ✓ ✓ ✓
Indigenous Participation ✓ ✓ ✓ ✓
Indigenous Study ✓ ✓ ✓ ✓
Learning ✓ ✓ ✓ ✓
Learning Skills ✓ NA NA NA
Learning self-care ✓ ✓ ✓ ✓
Lessons ✓ ✓ ✓ ✓
Literacy ✓ ✓ ✓ ✓
Media NA ✓ ✓ ✓
Numeracy ✓ ✓ ✓ ✓
Future Preparation
Idea of Future ✓ ✓ NA NA
Life Preparedness NA NA ✓ ✓
Interpersonal Skills
Collaboration ✓ ✓ ✓ ✓
Diversity NA ✓ ✓ ✓
Emotional Development ✓ ✓ ✓ ✓
Listening ✓ NA NA NA
Planning NA NA ✓ ✓
Problem-solving ✓ ✓ ✓ ✓
Resiliency ✓ ✓ ✓ ✓
Social Skills ✓ ✓ ✓ ✓
School Environment
Adult support ✓ ✓ ✓ ✓
Belonging ✓ ✓ ✓ ✓
Bullying (School Level) ✓ ✓ ✓ ✓
Community NA ✓ ✓ ✓
Discrimination (School Level) NA ✓ ✓ ✓
Fair Treatment ✓ ✓ ✓ ✓
Rules ✓ ✓ ✓ ✓
Safety (School Level) ✓ ✓ ✓ ✓
School support NA ✓ ✓ ✓
Teachers ✓ ✓ ✓ ✓
Wellbeing
Alcohol (School Level) NA ✓ ✓ ✓
Food Scarcity (School Level) ✓ ✓ ✓ ✓
Mental Health (School Level) ✓ ✓ ✓ ✓
Nicotine (School Level) NA ✓ ✓ ✓
Nutrition ✓ ✓ ✓ ✓
Physical Health ✓ ✓ ✓ ✓
Sleep ✓ ✓ ✓ ✓
Note:
Table 2 describes the constructs available by grade. If a checkmark is present, the construct is available in the grade. Constructs are on the leftmost column and summary constructs are denoted by bold underlining. If a construct is underneath a summary construct, it is included in that summary construct. If a construct has anonymous questions within it, the constructs are measured using school-level average responses (denoted as ‘(School Level)’) and are only available at the school-level; in all other cases, constructs use student-level responses.

2.3 Calculation of Construct Scores

After identifying which question belongs to each construct, construct scores were calculated by averaging the questions in each construct identified by factor analysis; this is called a ‘factor-based’ construct as the questions entering each construct are based on factor analysis.5 Summary construct scores are not calculated as the groupings are for reporting purposes only.

Construct scores range from one (1) to five (5) where 1 indicates a low score and 5 indicates a high score. Calculations exclude “don’t know” and missing responses. Scores are calculated in two ways depending on whether the constructs contain purely identifiable questions or contain anonymous questions. The mental health construct, for instance, has anonymous questions. For constructs that contain all identifiable questions, the student level construct scores are calculated for each student and then averaged to the school-level to get the school construct scores in Dashboards.6 If a construct includes anonymous questions, calculating the school construct scores involves two steps.7 First, the responses to each question in a construct are averaged at the school level. School construct scores are then calculated by taking the average of the resulting question scores for each construct. District and Province construct scores are calculated similarly. If a construct has no anonymous questions, district and province scores are equal to the average of all students’ construct scores. If a construct has anonymous questions, district and province scores are equal to the average of all school construct scores.

Students who have incomplete responses are included in the construct scores. A student is only excluded from a construct score if they have answered none of the relevant questions.

Table 3: Hypothetical Adult Support Calculation with Incomplete Responses

Question
Construct
Student Q26A S96 S110 S50 S99 Adult Support
1 1 2 3 4 5 3.0
2 5 4 5 5 5 4.8
3 4 3 NA 4 3 3.5
4 2 NA 3 NA NA 2.5
5 NA NA NA NA NA NaN

Table 3 illustrates how incomplete responses are accounted for using hypothetical responses and the adult support construct. Adult support has no anonymous questions, so a construct score is calculated for each student. Construct scores for students 3 and 4 are equal to the average of relevant responses completed. Student 5 does not have a construct score because they did not answer any of the relevant questions. The idea is to maximize the information available for analysis in the context where each question is related to the same underlying concept.8

2.4 Construct Robustness

Three results improve confidence in our methodology. First, construct fit statistics are very similar in randomly chosen districts. This suggests constructs scores are valid for districts: the questions identifying a construct provincially are consistently doing so in districts. Second, construct model fit indices were calculated and compared using data for the 2021/2022 and 2022/2023 school years. Similar model fit results were obtained suggesting questions are consistently interpreted the same way. Lastly, constructs passed several validity tests that were conducted, including face validity, convergent validity, and divergent validity tests. For example, learning outcomes are closely related to feelings of life preparation, adult support is strongly associated with belonging and fair treatment, and schools with higher alcohol/nicotine use report lower mental health and belonging.

The constructs were also peer reviewed within the ministry, and information sessions with districts will be conducted to obtain feedback and input from external stakeholders as part of the review process.

3 Applicability of Constructs

To illustrate where these measures are applicable, the following shows tests of how well constructs account for differences in numeracy and literacy marks and key student constructs. Each outcome is regressed on the set of constructs, and the (adjusted) R-squared is reported; R-squared is a statistic summarizing the percentage of variation explained by the constructs.9 The way to interpret test results is to take the variation explained as a proxy of how important the constructs are for explaining student outcomes.

3.1 Numeracy and Literacy

Numeracy and Literacy marks are measured using Foundation Skills Assessment scores for students in grades four and seven and Graduation Assessment scores for students in grades 10 and 12.

Figure 1: Constructs Explain 15-25% of Differences in Literacy and Numeracy Marks

Figure 1 illustrates constructs explain 15 to 25% of the differences in literacy and numeracy scores. The increasing trend by grade suggests the attributes constructs identify are more important in later grades.

3.2 Well-being

Figure 2 shows the relevance of these constructs as determinants of student well-being and success. Like the results in Figure 1, Figure 2 presents the R-squared from a regression of belonging or social skills on all other constructs at the student-level; for example, belonging is regressed on all the student-level constructs except belonging.

Figure 2: Constructs Explain 40-60% of Differences in Belonging and Social Skills

The constructs identify about 40% of the differences in belonging and social skill constructs for grade four and about 50 to 60% in grades 10 and 12. The greater explainability for non-academic outcomes, like social skills, is aligned with our expectations.10 These attributes are fundamentally psychological, and because they are directly related, they are more correlated.

Caution should be noted. These tests are approximations – the explanatory power of these constructs could be due to correlation with unobserved factors, such as cultural perceptions. The guide reader is advised to consider these results only to illustrate how the constructs can be helpful.

4 Conclusion and Recommendations

The purpose of constructs is to capture the breadth of student experiences present in the SLS and summarize them in a relevant way for users of the data. Statistical methods were used to define constructs capturing student experiences in school. These constructs are summarized further into five summary constructs for SLS Dashboard users, providing an overview of student development in key student success areas. Constructs are shown to be important determinants of some student outcomes, consistent with other research (Heckman et al., 2013).

Readers are recommended to use summary constructs to gauge where they need to focus their attention and efforts. Constructs are a more reliable and less error-prone way to summarize the information captured by SLS questions. Readers are recommended to use constructs to further understand how aspects of a student’s school environment are affecting their experience. Constructs can be used for research and to provide a descriptive overview. Readers can see provincial scores below and in the SLS Dashboard.

The goal of this guide is to provide an introduction to the constructs and how they could be of use for schools and districts. Based on the feedback received from districts and stakeholders, these tools will be continually refined to improve their reliability, validity, and use. For any technical questions, feedback, or guidance needed, do email us at:

5 Bibliography

Borghans, L., Duckworth, A. L., Heckman, J. J., & Weel, B. T. (2008). The economics and psychology of personality traits. J. Hum. Resour., 43(4), 972–1059.
DiStefano, C., & Hess, B. (2005). Using confirmatory factor analysis for construct validation: An empirical review. J. Psychoeduc. Assess., 23(3), 225–241.
DiStefano, C., McDaniel, H. L., Zhang, L., Shi, D., & Jiang, Z. (2019). Fitting large factor analysis models with ordinal data. Educ. Psychol. Meas., 79(3), 417–436.
DiStefano, C., & Morgan, G. B. (2014). A comparison of diagonal weighted least squares robust estimation techniques for ordinal data. Struct. Equ. Modeling, 21(3), 425–438.
Enders, C., & Bandalos, D. (2001). The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Struct. Equ. Modeling, 8(3), 430–457.
Heckman, J., Pinto, R., & Savelyev, P. (2013). Understanding the mechanisms through which an influential early childhood program boosted adult outcomes. Am. Econ. Rev., 103(6), 2052–2086.
Hu, L.-T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct. Equ. Modeling, 6(1), 1–55.
Li, C.-H. (2016). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behav. Res. Methods, 48(3), 936–949.
Revelle, W., & Condon, D. M. (2018). Reliability from alpha to omega: A tutorial. In PsyArXiv.
Revelle, W., & Zinbarg, R. E. (2009). Coefficients alpha, beta, omega, and the glb: Comments on sijtsma. Psychometrika, 74(1), 145–154.
Savalei, V. (2010). Expected versus observed information in SEM with incomplete normal and nonnormal data. Psychol. Methods, 15(4), 352–367.
Shi, D., & Maydeu-Olivares, A. (2020). The effect of estimation methods on SEM fit indices. Educ. Psychol. Meas., 80(3), 421–445.
Shi, D., Maydeu-Olivares, A., & Rosseel, Y. (2020). Assessing fit in ordinal factor analysis models: SRMR vs. RMSEA. Struct. Equ. Modeling, 27(1), 1–15.
Thompson, B. (2004). Exploratory and confirmatory factor analysis (B. Thompson, Ed.). American Psychological Association.
Voorhees, C. M., Brady, M. K., Calantone, R., & Ramirez, E. (2016). Discriminant validity testing in marketing: An analysis, causes for concern, and proposed remedies. J. Acad. Mark. Sci., 44(1), 119–134.
Widhiarso, W., & Ravand, H. (2014). Estimating reliability coefficient for multidimensional measures: A pedagogical illustration. Review of Psychology, 21(2).
Xia, Y., & Yang, Y. (2019). RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods. Behav. Res. Methods, 51(1), 409–428.

6 Provincial Construct Scores

  1. As such, all of a student’s experience is captured – not just their most important attributes.↩︎

  2. In particular, constructs are considered related if their loadings are not weak (greater than 0.4) and their question content is highly related. Following Hu & Bentler (1999), model fit us evaluated using RMSEA < 0.1, Cronbach’s Alpha > 0.7, CFI and TLI > 0.9; as guidelines, RMSEA is minimized and Cronbach’s Alpha, CFI, and TFI is maximized. Full-information maximum likelihood was used to account for missing responses (Enders & Bandalos, 2001; Savalei, 2010). To test robustness, McDonald’s Omega was estimated as Cronbach’s alpha can be problematic (Revelle & Condon, 2018; Revelle & Zinbarg, 2009). The results for McDonald’s Omega and Cronbach’s alpha are exceptionally similar. Similarly, loadings and model fit are robust to WLSMV models accounting for categorical responses (DiStefano et al., 2019; DiStefano & Morgan, 2014; Li, 2016; Shi et al., 2020; Shi & Maydeu-Olivares, 2020). Following literature recommendations, this criteria acts as guidelines rather than cut-offs since they can be problematic – especially when using school rather than student variation – to assess which models are superior. Please reach out to if you have further technical questions.↩︎

  3. Factor loadings for one or two question models are not as accurate since the models are not as well identified. Factor analysis relies on the correlation between multiple questions to estimate factor loadings.↩︎

  4. School-level only constructs, like mental health, have less meaningful factor loadings since they use school average question relationships instead of student-level. The theoretical framework is relied on more heavily for these constructs.↩︎

  5. Averaging question responses is a commonly used method to define constructs (Borghans et al., 2008; Heckman et al., 2013). While equally-weighted averages are used for descriptive statistics like those present in the SLS Dashboard, factor scores are defined for statistical inference research. These scores more closely approximate true underlying concepts (Heckman et al., 2013; Thompson, 2004). However, they are also difficult to interpret on their own. Where a factor-based (average) construct of mental health explains how school A students are doing overall, a factor score of mental health only explains how school A students are doing relative to all other schools. Descriptive factor-based (average) construct scores are therefore presented in the SLS Dashboard.↩︎

  6. The score calculation methodology implicitly assumes a linear scale in Likert responses. This appears reasonable since non-parametric regressions on survey question responses are often approximately linear.↩︎

  7. The assumption here is that responses are random samples of each question’s attributes. The constructs are then equally-weighted averages of these attributes for a school. The alternative would be to average over all numeric responses, weighting the values by the number of responses. This method was not chosen because it does not consider the value each question provides, weighting instead by the number of responses.↩︎

  8. Construct score calculations account for incomplete responses in a very similar way to how questions are grouped into constructs. Construct groupings account for incomplete responses using full information maximum likelihood.↩︎

  9. Reporting R-squared as a measure of construct relevance is considered to be an approximate test. The test assumes linearity, and more importantly, the variation explained by the constructs could be caused by correlation with non-included factors. This test suggests, but does not prove, validity and usefulness of the measures.↩︎

  10. The constructs are similarly representative of other outcomes. As the reader may expect, attributes like “resiliency” and (school) “indigenous participation” are affected more by other factors, and consistently, the constructs explain less variation for these outcomes. ↩︎